Get Out of Jail Free — OMNI Early Prisoner Release in Washington State

The 6th anniversary of the Washington State criminal tracking system bug resulting in early prisoner release offers an opportunity to reflect upon computer system defects, human error, process flaws, organizational mistakes, and the best principles and practices for solution delivery in the IT industry. In this blog and my upcoming book**, Bugs: A Short History of Computer System Failure**, I will chronicle some important system failures in the past and discuss ideas for improving the future of system quality. As information technology becomes increasingly woven into Life, the quality of hardware and software impacts our commerce, health, infrastructure, military, politics, science, security, and transportation. The Big Idea is that we have no choice but to get better at delivering technology solutions because our lives depend on it.

On 22 December 2015, Washington state governor Jay Inslee announced to the public that the Washington State Department of Corrections (DOC) had a computer bug in its Offender Management Network Information (OMNI) criminal tracking system that was incorrectly calculating the release date of prisoners in violation of the law, that the system bug had existed for more than twelve years, and that some DOC staff had been aware of the bug for almost three years. Governor Inslee immediately ordered the halt of all prison releases until manual calculations could verify the correct release date; he promised delivery of the system fix within thirty days and also appointed two former federal prosecutors, Carl Blackstone and Robert Westinghouse, to conduct an independent investigation into the matter. In the weeks following the announcement, the state and public learned that nearly 3,200 convicted felons had been released too early over the past decade, and that at least two of these individuals had recently committed violent crimes including homicide and murder. Washington state and its taxpayers had spent approximately $72 million on development of the OMNI system including separate contracts with IBM and Sierra Systems; the legal liability faced by the state was also estimated in the millions of dollars. This essay explores the key business, technology, and human factors that contributed to the system defect and DOC’s organizational failure to prioritize its fix.

The Washington state legislature created the Department of Corrections in 1981. DOC is responsible for managing adult prison facilities and supervising adult offenders residing in the community on probation. The department operates twelve prisons in the state of Washington; ten prisons house male inmates and the other two facilities house female inmates. There are approximately 16,000 inmates incarcerated in these prisons. DOC employs about 8,000 individuals and has an annual operating budget of $1.7 billion. It costs the state of Washington about $25,000 annually to take care of a minimum security inmate and about $50,000 annually to take care of an elderly, special needs, or maximum security inmate.

Once an individual is arrested for a crime, they may be either released on bail or detained in a local county jail pending trial. If the defendant pleads guilty or is convicted, then a judge imposes a sentence. If imprisonment exceeds one year, then the miscreant is transferred from county jail to DOC custody to serve the sentence in prison. DOC becomes responsible for determining the individual’s release date from prison. Offenders can reduce their sentence through an incentive program known as “earned release time”. These credits can be earned for good conduct, being free of infractions, and participating in work and education programs. For most criminal offenses, earned release is capped at one-third of the offender’s sentence. Washington state adopted a series of laws in 1995 which imposed mandatory minimum sentences for certain serious offenses and added sentencing enhancements if an offender uses a firearm, a deadly weapon, or engages in crime with a sexual motivation. For example, robbing a business with a gun may result in a five-year enhancement on top of the base sentence for robbery. The enhancement must be served in its entirety without any reductions.

On 3 July 2002, the Washington state Supreme Court ruled In Personal Restraint of King that DOC must reduce base sentences by crediting inmates for “good time” earned while still in county jail and not treat the county jail time as part of the enhanced sentence. Prior to this verdict, DOC credited offenders only for the good time they earned while in prison. The King decision instructed DOC to begin running the enhancement “clock” on the day the offender was transferred to State custody, applying the time in local custody against the criminal’s base sentence. The court had suggested a simple 2-phase formula in which the enhancement or mandatory minimum sentence “clock” starts first and is allowed to complete without interference of credits and other factors; in the second phase, the inmate would serve the base portion of the sentence, and the formula would adjust this “clock” by the earned good time credits and time spent in pre-sentence detention. Unfortunately, the DOC did not use such a straightforward formula; instead, it mis-interpreted the court ruling, hatched its own complex calculation methods, and continued its practice of sequencing the offender’s enhancement period before the offender’s base period and somehow still credited the amount of jail time served against the enhancement. It also began crediting any jail good time earned to the base sentence before calculating prison good time. At the time of King decision, DOC used its legacy mainframe-based Offender Based Tracking system (OBTS), and in March 2003, DOC reprogrammed OBTS based on incorrect Requirements and formula in the original work request submitted by Janice McMann, then the DOC Records Manager. This OBTS defect was also migrated to the new web-based OMNI system. The illustrative sentencing calculation example below come from the Westinghouse-Blackstone Investigation Report.

Illustrative OBTS Calculation Example from Blackstone-Westinghouse report

OBTS was acquired from the State of Florida in 1983 and implemented in 1984. OBTS was a traditional mainframe system written in COBOL, and it had been extensively modified to meet the changing requirements of Washington state sentencing laws. It could not keep up with ongoing needs especially effective tracking of parolees; this inability had already cost the state tens of millions in lawsuit payouts. The DOC conducted a feasibility study in 1998 on the replacement of OBTS; the study recommended a phased system development approach. In April 1999 a RFP process began, and bids from several vendors were evaluated including Deloitte, IBM, Oracle, and Sapient. OMNI’s scope was ambitious and complex; it would consist of multiple modules including prisoner confinement status, rehabilitation curriculum, treatment, sentencing, transportation, discipline, case management, bed capacity management, and community supervision for parolees. Later that year, IBM was awarded the $58 Million contract for OMNI. Phase 1 of the project commenced in November 1999; it was delivered, but remained buggy. Work began on phase 2 in July 2001, but it was never deployed. Work commenced on phase 3 in December 2005, but never finished. Fast forward to August 2006. OMNI was already two years behind schedule and $6 Million over budget. Earlier in 2006, the system had crashed for four days leaving the DOC unable to transport prisoners or otherwise generate any information. In September 2006, the contract was transferred from IBM to Sierra Systems. Sierra assumed full responsibility for successful execution of the OMNI project including project management, specifications, design, and development. Sierra delivered OMNI in August 2008 for the cost of $14 Million. Although OMNI has been iteratively improved over time, continuing legislative changes to State’s sentencing laws and ongoing mechanical calculation defects have resulted in DOC IT continuing to modify the Sentencing Structure and Time Accounting (SSTA) module calculations.

Now fast forward from 2008 to December 2012. King county resident Matthew Mirante was then notified that DOC intended to release Curtis Robinson who had been convicted of stabbing Mr. Mirante’s son multiple times in November 2011. Mirante suspected that the release date was premature, performed his own calculations, and then contacted Steve Eckstrom, the DOC Victim Services Program Manager. According to several DOC staff, this was the first evidence that the agency’s OMNI system was still computing prisoner release dates incorrectly in light of the King decision. Eckstrom verified the discrepancy himself and became concerned that DOC had not been calculating offender early release dates correctly. He called Wendy Stigall, DOC Records Manager to advise her of the problem, and Stigall said she would follow up with IT and the Attorney General’s office. On December 7, 2012, Eckstrom also informed Ronda Larson, the Assistant AG. Stigall and Larson discussed the problem further that day, and then Larson published an infamous memorandum on the matter by email, notified Stigall and her supervisor, Attorney General Paul Weisser, and recommended a hand calculation in the case of Mrs. Robinson, but not for all offenders. In the memo, she recognized the potential legal and financial liability if other offenders were released early and committed further violations, but believed the cost of so many manual calculations was too high, the process for changing OMNI too slow, underestimated the risk to just “hundreds of sentences”, and did not take the step of stopping releases to protect public safety until offenders’ release dates could be verified by manual calculation. Robinson’s release date calculations are listed below.

Curtis Robinson Release Date Calculation

Later on 7 December 2012, Stigall forwarded the Larson memo onto DOC IT staff including Sue Schuler, a business analyst who regularly collaborated with stakeholders. On December 27, 2012, Stigall submitted a formal IT change request marked with priority “ASAP”, and a week later on January 3, 2013, Schuler began a consultation to understand the nature of the problem, the solution needed by the user, and an estimate of the time to fix the problem. Schuler did not complete the consultation until March 25, 2013, almost three months after it was first assigned to her. Schuler later claimed to not recollect Stigall communicating the urgency of the problem, suggested that she had not received all necessary information needed from Stigall to proceed, and also believed, per the Larson memo, that the system bug only affected a small number of inmates. However, according to evidence submitted to the state’s investigation, Stigall had provided a spreadsheet with multiple examples of how good time should properly be calculated. Schuler eventually forwarded the change request and consultation form onto Mark Ardiel of Sierra Systems; Ardiel estimated that fixing the bug would take just 16–20 hours of programming work. On April 3, 2013, the change request was forwarded onto the OMNI Triage Team for approval.

The responsibility for the King fix was now firmly in the hands of IT. OMNI system releases are overseen and managed by DOC while programming support is handled by Sierra. Normal OMNI releases are typically completed every eight weeks as scheduled maintenance or “M” releases. Urgent items or “hot fixes” are done on an as-needed basis between M releases. The backlog of features and defects are tracked in ClearQuest; backlog items are assigned a priority ranging from level 1 (“Critical Impact”) to level 4 (“Minimal Business Impact”) and queued for a particular M-release. Approved changes are then assigned to a software engineer. Once the programmer has completed their work, then the item is sent to the testing team and when testing is successfully completed, the item is scheduled for delivery in the M release. The OMNI service team meets twice a week to review the status of pending items in an upcoming M-release. There are no minutes or recorded notes of OMNI team meetings. These meetings were attended by Dave Dunnington, the IT Business Manager for Prisons, all IT Business analysts, Deepak Sandanan, the lead DOC tester, Trang Nguyen, the lead DOC programmer and Mark Ardiel of Sierra. Only Messrs. Dunnington and Sandanan could move items to a later release date. Schuler originally marked the King defect item (“OMNI00024910”) with a priority rating of 2 (“Serious Impact”), but in February 2014, David Dunnington demoted the item to level 3 (“Moderate Impact”) and it remained as such until it was delivered in January 2016. Schuler had also requested that the King fix be scheduled for release in M34 intended for September 12, 2013 since the two earlier releases were already filled with other allegedly important features and fixes. In June 12, 2013, Dunnington reassigned the King fix from M34 to M35 without justification. Although IT staff are supposed to enter notes into the Clearquest system explaining status changes, with rare exceptions, no explanations or reasons were entered for the delays in the King item. On July 10, 2013, Stigall followed up with Schuler to prioritize several items in the IT backlog including the King fix. On August 15, 2013, Stigall held a meeting with all DOC record managers and discussed the King fix; Secretary Bernard Warner was present for part of the meeting and the fix’s priority was repeated to all the stakeholders. On September 27, 2013, Deepak Sadanandan marked the King fix as “MUST FIX”, but somehow delayed it until M37. Later on October 1, 2013, Dunnington rescheduled the King fix to M38. Ardiel then picked up the ticket and finally began to work on the King fix for the first time; he spent 80–100 hours initially working on the item, but was later told to redirect his efforts elsewhere as he encountered issues and could not get additional information from Stigall. Over the next two years, Dunnington and Sandandan would delay the King fix another half dozen times. In February 2015 — September 2015, Ardiel took paternity leave, and the backlog item was further delayed several M-releases. Even when Ardiel returned, the fix continued to languish in the IT queue. On November 2, 2015, the new DOC CIO Ira Feuer met with Stigall and others as a part of on-boarding and connecting with key DOC stakeholders. Stigall mentioned the King fix and its stubborn delays; Feuer recognized the importance and investigated the matter, personally meeting with Dunnington later that day. Work finally resumed on the King fix the next day, and Ardiel spent about 300 hours during November and December; testing initially failed and required further development by Ardiel and others until the work was successfully completed in January 2016.

During a DOC staff leadership meeting on December 15 2015, Stigall again raised the alarm about the ongoing delays for the King fix, and Secretary of Corrections Dan Pacholke then learns of the problem for the first time. On December 16, DOC leadership met with the governor’s staff, and on December 17, Governors Inslee is briefed on the problem and also learned about it for the first time. Over the next few days, the governor meets with leadership in several departments including Communications, Corrections, IT, and Legal to coordinate a proper response. On December 22, Governor Inslee publicly disclosed the problem. On January 12, 2016, the OMNI M49 release including the King fix was finally delivered to production.

The OMNI system defect had serious consequences for the residents and government of Washington state. Over ten years, almost 3,200 inmates were released earlier than the statute allowed by a median of 59 days according to DOC analysis. 116 of those freed were required to return to prison after they were accused of crimes or parole violations after their release. At least 60 of the early released inmates were taken back by the state into custody. Two deaths were connected to criminals who should have been in prison at the time, and other crimes also may have been committed by those who should have been in state custody. Among these violent delinquents was Robert Terrance Jackson. Jackson was released six months early in 2015 and crashed his car while drunk into that of his girlfriend, Lindsay Hill, a 35-year old mother of two in November 2015. Hill was thrown from her car and suffered a fatal head injury. According to the police investigation, Jackson had been dating Hill, the relationship was abusive, and Jackson was charged with vehicular homicide. The other violent offender was Jeremiah Smith. Smith was released three months early from prison on May 14 2015 and then two weeks later, shot Caesar Medina on May 26. The DOC confirmed his release date should have been August 10, 2015. Medina’s family sued the state for damages and won a $3.25 million settlement. In addition to the threat to public safety, the DOC’s mismanagement of the King fix created large financial costs for Washington taxpayers because of the potential legal liability borne by the state. Although crimes were committed by the improperly released offenders and not by the state itself, Washington law allows claims for “negligent supervision” that make the state easier to sue than other states. Furthermore, the state waived its sovereign immunity to tort claims back in 1961, allowing the state to be sued just like any person corporation. Finally, Washington state is not protected by limitations on claims that usually apply in other states such as damage caps and special procedural requirements that make it more difficult to sue.

Two official government investigations were conducted into the King error. The Blackstone-Westinghouse inquiry was ordered by governor Inslee and another was led by senators Mike Padden and Steve O’Ban on the state Senate’s Law and Justice committee; both reports were based on interviews with dozens of key witnesses and the review of hundreds of documents. Synthesizing the conclusions and recommendations from both reports, one can draw several lessons useful for technology and business leaders.

The People and Leadership of an organization are the basis for its success (and failure) and the key capability for adapting to change and challenges. DOC Secretary Bernard Warner’s tenure from July 2011 to October 2015 was marked by a number of acute problems highlighted in the state senate report. First, there was excessive leadership turnover amongst different divisions within the DOC, especially in IT. Six different individuals held the role of CIO under Warner; three only served for no more than three months each. Without leadership continuity, upper IT management did not properly monitor daily operations, and mid-level employees failed to communicate with their peers or question executive policy decisions. Second, the senate report characterized Warner as aloof and distant, he frequently traveled to out-of-state and international conferences on prison management, and he also added layers of people between himself and department staff who managed day-to-day operations. He was a poor communicator, an opaque supervisor who neither expressed a vision nor set actionable goals for his team, and someone who lacked interest in the details and oversight of IT. Oddly, Warner even refused to share his schedule with his own chief of staff or with Mr. Pacholke, his deputy, citing safety concerns. Third, Warner had an intimate relationship with someone on the governor’s staff that created a conflict of interest and may have contributed to the delay in governor Inslee learning of problems within DOC. Fourth, Warner was also directly responsible for prioritizing an ill-advised digital transformation project code named STRONG-R that consumed scarce IT time and resources in 2014 despite ongoing budget pressures. He also ordered that this project be handled by a single external vendor, Assessments.com, that was owned by a personal friend, Sean Hosman, who was a convicted criminal and someone troubled by substance abuse addiction. Former DOC CIO’s Kit Bail and Ira Feuer both identified STRONG-R as a contributing cause to the department’s distractions in the Senate government reports. Corrective action for poor executive leadership should include instituting peer review by subordinates; both reports also suggested the establishment of an independent Ombudsman to resolve DOC issues and a confidential performance reporting mechanism so that accurate information from whistleblowers could reach the governor’s office.
The DOC’s IT Governance Process was simply broken. Middle management should have detected and immediately corrected the problem when evidence of the miscalculation arose in December 2012. Unfortunately, the King fix was treated by Dunnington and other IT colleagues as a routine software maintenance defect; moreover, scheduling of the King fix was repeatedly delayed a staggering 16 times according to the Blackstone-Westinghouse report. Such organizational inertia results from cultural issues that are also larger than just one individual or one team. Stigall waited almost 3 years to personally intervene and seek executive involvement despite the clear risk to public safety. Several senior management officials including the DOC Risk Management Director (Kathy Gastreich) and DOC CIO (Doug Hoffer) claimed to have no recollection of the King issue being raised despite being directly informed by Stigall. Both reports recommended that the business should set priorities — not IT. Delays to urgent items should have also been documented and justified to business stakeholders along with a failsafe mechanism that notified executive stakeholders if such items were repeatedly delayed. Finally the Blackstone-Westinghouse report recommended an independent auditor to review the governance process regularly and verify that improvements were assimilated properly.
Both reports also pointed to the Complexity of the Requirements for the Sentencing Structure and Time Accounting (SSTA) module as another contributing factor to the King error being passed as an inherited disease from OBTS to OMNI. Applying William Ockham’s philosophical dictum to simplify not only scientific models, but their underlying assumptions suggests that the sentencing statutes in Washington themselves must be simplified in a manner that neither reduces punishment nor compromises public safety.
The governor’s report mentions Budget cuts across Washington state government in the wake of the 2008 financial crisis, and one could make an argument that a lack of Resources prevented a concerted, dedicated effort to fix the King error and contributed to the delays. As far back as 2011, the Seattle Times reported that Washington state was facing a $5 Billion budget deficit and was exploring options for saving money including shorter sentences for some non-violent offenders (e.g. those who have not committed sex offenses, murder or certain drug offenses). A corrective action was to review budgets and staffing of DOC IT and Records units to ensure adequate service levels for maintaining public safety. Another action item was to ensure that the vendor Sierra Systems had additional staff to fix critical problems in OMNI.
Organizations that rely primarily upon automation and computer systems for critical business processes should also have a secondary approach for redundancy and resilience in the case of system failure. The problem is that the DOC did not even follow its own SOP and protocol for manual calculation of release dates. Both reports recommended the requirement of a DOC-wide manual computation for all prisoner releases until the King fix for OMNI could be implemented and delivered.
Both reports also recommended that the Attorney General opinions to the DOC should be subject to supervisory review and approval. Ronda Larson’s memo represented poor legal advice and judgement; it created liability for the state, risked public safety, and was also used to justify the slow IT response to the King error.
Finally, both government reports concluded that the DOC should emphasize public safety as its most important statutory duty to all employees and make clear how this mission should impact daily decisions.

As so often happens when crises happen in large government organizations, few heads rolled and accountability was difficult to assign in the wake of the King error. Although DOC Secretary Dan Pacholke resigned in February 2016, and CIO Ira Feuer resigned in March 2016, few of those directly involved with and responsible for the King error were impacted. Wendy Stigall remains Statewide Records Manager for DOC, and Kathy Gastreich kept her job as Risk Management Director. Washington state’s incident with electronic management of prisoners is also not isolated. A similar incident happened to California in 2011 when 450 violent offenders were accidentally put on a “non-revocable parole” list, and also in Michigan during 2003–2005 where its DOC’s computer system for tracking inmates allowed the early release of prisoners, did not audit adjustments to prison sentences, did not enforce authorization policies, and permitted the prematurely early release of some inmates when their parole revocation hearings occurred improperly.

Enjoy the article? Follow me on Medium and Twitter for more updates.

References