Health Care Pricing: Can big data help us here?

This morning I read an article in the Economist magazine January 12, 2019 edition titled, “Shopping for a Caesarean”. This article summarizes the challenges that we face in the US around pricing for medical procedures. The true cost of medical procedures is lost in reams of arbitrary pricing algorithms.

In an era of “big data” convoluted pricing presents a great irony. We have data that corresponds to nearly every other facet of our lives. This data helps businesses predict consumer behavior in order to market the right product to consumers at the right time.

In the health care industry, hospitals don’t have to predict consumer needs. Rather, consumers will purchase a procedure when they are sick and/or under “duress” (the word used in the Economist article). They aren’t likely to shop around. This “duress” allows hospitals to use creative pricing, make deals with insurers, and do all sorts of tricks that conceal the true cost of healthcare.

The Economist article argues that price transparency is the first step, but that it won’t solve the problem because of the “duress” faced by those in need of care. What is needed is a big picture look at pricing for all of us to see when we are not in duress. This way we can identify who exactly is benefiting from these gross inefficiencies. We need “big data” for the masses. We need “big data” that will improve the standard of living for average folks just like we have “big data” that helps businesses market products. However, as long as the medical industry profits greatly from hidden pricing algorithms, they have little incentive to share their secrets and drive more efficiency into the marketplace.

Originally, this lack of transparency was probably not intentional, but now that it generates so much profit for the healthcare industry there is very little incentive to do anything about it. We need more than transparency around pricing for each procedure; we need “big data” algorithms that will allow us to untangle our current pricing mess.

Amazon Athena – What?

If you’re like many IT professionals who’ve had anything to do with large amounts of data, you’ve become immune to the phrase ‘big data’. Mostly because the meaning behind that phrase can vary so wildly.

Processing ‘big data’ can seem out of reach for many organizations. Either because of the costs in infrastructure required to establish a foothold on this front or because of a lack organizational expertise. And since the meaning of ‘big data’ can vary so much, you may find that you’re doing ‘big data’ work and then ask yourself, “Is this big data?” Or an observer can suggest that something is ‘big data’ when you know full well that it isn’t.

With my own background in data, I’m ever curious about what’s out there that can make the threshold into ‘big data’ seem less insurmountable.  Also, I’m interested in the security considerations around these solutions.

In the last week or so, I’ve gotten more familiar with AWS s3 buckets and a querying service called Amazon Athena. Here’s the truly amazing thing. You can simply drop files in an s3 bucket and query them straight from Amazon Athena. (There are just a couple steps to go through, but they are mostly trivial.) And for the most part, there’s not much of a limit for how much data you can query and analyze. You can scan 1tb of data for $5. What? That’s right. And you didn’t have to set up servers, database platforms, or any of that. I’ll be exploring Amazon Athena more and more over the coming weeks. If you have an interest in this sort of thing, I suggest you do the same.

One note: Google has something similar called BigQuery, so that might be worth a look as well. I’ve explored BigQuery briefly but I keep coming back to various AWS services since they seem to be holding strong as a dominant leader in emerging cloud technologies. But as well all know, the emerging technology landscape can change very quickly!