Gearing up for an annual physical recently, I stoically filled out 12 pages of survey data and disclosure forms. One of the forms in this bureaucratic slog was a mental health survey masquerading as a “patient health questionnaire.”

If you have been to a hospital recently, you will be familiar with this form. In a series of leading questions, it asks how often you “experience feelings of hopelessness, despair or depression” and other behaviors purported to flag suicidal tendencies.

Medical providers hoover up data almost as fast as Google, and meeting their demands can feel like dropping pebbles into a voracious black hole. Much of the information we regurgitate has no apparent bearing on our diagnostic and clinical experiences. Nor do medical providers seem to retain any record of information provided in previous visits.

What do they actually do with all of our personal information?

An excellent feature by Maia Szalavitz in the October issue of Wired magazine illuminated one disturbing answer to that question. Szalavitz’s story reveals how mental health surveys and a large agglomeration of both public and private data have been co opted to feed a secretive algorithm that is now the front line in the industry’s battle against opioid addiction.

That particular algorithm is lodged within commercial software called Narxcare, which is advertised as an “analytics tool and care management platform.” More accurately, it is a proprietary database owned by a company called Appriss.

Narxcare is sold to doctors, pharmacies, and hospitals. It complements Appriss’ legacy products – databases that offer a unified view of balkanized public data like court records, criminal records, and SSN-linked identifiers. Employers, law enforcement, and other institutions use those products to screen applicants, search criminal histories, and provide continual monitoring for convictions.

Like most enterprise software companies, Appriss is owned by private equity investors – in this case Clearlake Capital Group and Insight Partners. This fact should help clarify the core values under which the company operates.

For the past twenty years the US Department of Justice has spent hundreds of millions building and maintaining prescription drug databases that track scripts for controlled substances, giving authorities a real time lens into the market for particular pharmaceuticals. Every state except Missouri now has one of these prescription drug monitoring programs (PDMPs).

While individual state registries differ in many parameters and protocols, one company is directly involved in the management of most of them. It created enough interoperability to make the PDMPs function as the equivalent of a national prescription drug database. That company is Appriss.

Narxcare offers several capabilities for the fight against opioid addiction. The software mines state PDMP data for red flags associated with drug seeking behavior. These include certain combinations of pharmaceutical scripts, visits to a large number of pharmacies, and excessive distance traveled to receive care (an indicator of “doctor shopping”).

The dataset underlying Narxcare includes PDMP data, medical claims, EMS data and criminal justice records. That data is summarized by a series of three digit numbers that reflects past exposure to opioids, sedatives and stimulants and the number of currently outstanding scripts for each. As an enhancement, Narxcare clients can also access a predictive machine learning product that generates an Overdose Risk Score (ORS) for each patient.

Only Appriss knows how the ORS score is generated, but history offers some tantalizing clues. In 2005, as concern about opioid addiction ramped up, Lynn Webster published the Opioid Risk Tool (ORT) as a publicly available framework for early identification of atrisk patients.

Webster, the former president of the American Academy of Pain Management, gathered extensive research that linked addiction to opioids (and other drugs) to family addiction history and psychiatric disorders, including obsessive compulsive disorder, bipolar disorder, schizophrenia, and depression.

Webster built the ORT as a series of questions meant to identify these correlative risk factors for addiction. The ORT was widely adopted and its emphasis on correlation data was perpetuated in Narxcare. Patients deserve to know this when filling out “patient health questionnaires.”

Chronic pain affects one of every five people. Roughly 70% of US adults have been prescribed opioids. An estimated 0.5% suffer from opioid use disorder. Pharmaceutical replacements for the opioids and sedatives on the market today appear to be decades away.

The ORT and Narxcare may prove extremely helpful in identifying the addiction prone, sooner rather than later. However, there are risks in relying too heavily on predictive correlations and mystery algorithms. According to one study, fully 20% of those most likely to be flagged as doctor shoppers are cancer patients.

Appriss has repeatedly stated that Narxcare scores are not intended to replace a doctor’s diagnosis. However, physicians ignore those three digit numbers at their peril. Nearly every state uses Appriss to manage its PDMP, and most legally require physicians and pharmacists to consult them when prescribing controlled substances on penalty of losing their license.

AI and machine learning algorithms promise to automate and scale information intensive fields such as medicine in ways that we don’t fully appreciate. However, their adoption should be accompanied by the same kind of public disclosures and debate that scientific papers receive. Mystery algorithms should not assign a scarlet “A” to pain patients without a more transparent process that includes opportunities for appeal.