A few months ago, we introduced the Column API which exposed free structured access to public notices for the first time. Today, we are expanding the Column API beyond public notice with a new dataset born out of a recently enacted federal disclosure requirement: Healthcare Provider Negotiated Rates.

Our team believes that simply disclosing public information isn’t enough; you have to make that information useful. This has been our approach to public notice. And now, we are tackling a new challenge and a new dataset.

As of July 1st 2022, the federal government introduced a new disclosure requirement: all health insurance providers must disclose their negotiated rates with all healthcare providers across the US. However, just as local public notice laws do not help the modern world easily access public notice information, the new Transparency in Coverage mandates have not led to easy access to healthcare rate information for the public.

Given the steep fines that penalize non-compliance, nearly every insurance company has scrambled to upload its pricing information to the web. However, these data files are in formats so arcane and so challenging to deal with that it is effectively impossible for anyone to actually make use of this information. Some are sharing tens of thousands of compressed CSVs behind bandwidth-limited URLs. Some are sharing broken JSON files. And some share files on hidden pages, buried deep within the nested sub-menus of their sites.

In aggregate, the impact of these measures taken by these insurance companies is summed up well by Sabrina Corlette, a researcher on the subject at Georgetown, in a quote to NPR:

“There is data out there; it’s just not accessible to mere mortals”

Good news! Our API is not a mere mortal. We’ve spent a lot of engineering time figuring out how we can build effective pipelines that take the petabytes of data being poured out into the internet on a monthly basis and condense them into a practical and useful form. And now, we’re sharing that work with the internet to see what folks can do with access to this data.

What have we done? Explore for yourself.

We’ve removed the burden of scraping, structuring, formatting, and compressing large swaths of negotiated rate information and opened it up to the public.

This data is available for free on our documentation site at: docs.column.us.

If you want to start playing around with this yourself, try pasting a few of the following sample queries into the shell below:

1. SELECT COUNT(*) FROM 'providers'; — get a list of all of the providers with prices listed for them
2. SELECT COUNT(*) FROM 'prices; — see how many insulin prices we have listed
3. SELECT * FROM 'prices' LIMIT 5; — see samples of the pricing data format

We’re committed to making this dataset and others like it useful. And if that resonates with you, we would love to work with you or your organization. We certainly recommend signing up for our newsletter below to stay in the loop on the great work our team will ship throughout the year.