Determining ambulance destinations when facing offload delays using a Markov decision process

When emergency departments (EDs) are crowded and cannot accept incoming ambulance patients immediately, paramedics commonly continue to provide patient care until an ED bed becomes available. This delay in transferring a patient to the ED is referred to as ambulance offload delay (AOD). AOD is a pressing problem for Emergency Medical Services (EMS) as it prolongs the time before paramedics are available to respond to other calls. This can negatively affect ambulance availability and patient safety. The objective of this study is to develop an ambulance destination policy to mitigate AOD, allowing patients to see physicians sooner, and returning ambulances to service more quickly. We formulate a discrete time, infinite-horizon, discounted Markov Decision Process (MDP) model to determine when it is advantageous to send appropriate patients to out-of-region EDs, which have longer transport times but shorter offload times. Based on the MDP model, an optimal ambulance destination policy is constructed using the policy iteration algorithm. A computational study is applied using 12-months of data from an EMS provider which experiences AOD regularly. We find that the optimal policies can significantly reduce AOD, time to bed for patients, and out-of-service time for paramedics at the expense of increased ambulances travel distances. The model can be generalized and used as a decision support tool for EMS systems to mitigate the impact of AOD on their operations.