Today I’m incredibly excited to show off a project I’ve been working on, but first a quick back story.
Figure 1. Ray tracing of my ankle using set.health tech. The loose fragments along the right are the displaced pieces of the break.
I have been writing software since I was 10 years old. Back then, I barely knew what I was doing, gluing together parts of different videogame websites I loved.
I do remember though the first moment my brain clicked and realized that programming was more than putting things together, but gives you the opportunity to build new things from the ground up. Since then, it was all about the product, building new things.
When the first iPhone was released, I was fascinated with the idea of building something that could be used by thousands of people. So I joined an open-source project to build videogames. I learned a lot! Computer graphics, teamwork, and how a game engine works under the hood, someone could say I got my CS “degree” during that time.
This allowed me to build and ship a couple of games, one of them very successful, I worked tirelessly polishing the gameplay, the design of every little aspect, adjusting the difficulty and the controls obsessively. It was all about the product, I didn’t care much about the code, it was a mere tool to accomplish something, but then I fell in love.
I fell in love with software engineering, I rediscovered the complexity of building software, the software itself. The developer experience of people using my code, how an API will be able to scale and be future proof, optimizing processes so more developers can work without stepping on each other. Since then, I have worked building SDKs and APIs, created Gin, and Ionic and Stencil using web technologies.
During all this time I had no time to stop and think, not a single week without coding, week after week, then year after year. Thinking more about the “Why”, rather than the “How”. However, back in October 2019 something happened. One night some friends and I went out for drinks close to my office in Kreuzberg, a vibrant district of Berlin.
That night I broke my ankle.
Suddenly, everything I was doing, I was working on stopped abruptly. For 14 days I “lived” in that hospital of Kreuzberg, waiting for a surgery until the swelling shrinks. When I was able to calm down, it was not that bad, I found time to read those books I always wanted to, and books I would have never read, reconnected with people, wrote more, designed a tattoo for my ankle and even learned some German: “Ich möchte gerne einen Kaffee mit Milch ohne Zucker”. A time to think and disconnect the autopilot.
The worst was to come, even though the doctors assured the surgery went well, I had the feeling something was not good. After dealing with a very stressful process, I managed to get my health data onto a CD, bought an external CD reader on Amazon and asked for a second opinion.
I was told to come back immediately to Spain (where my family is from). Indeed, there were problems, the articular space was not respected, nerve damage because of the suture, an unrepaired tendon and the syndesmosis screw not placed in the optimal position. I was scared, tired, and depressed, but after the second surgery I was able to find some peace again.
Without deadlines, I opened my laptop and remembered those old days where I used to write game engines and build things. After some research, I tried to visualize my health data, writing all the software from scratch.
It was a lot of fun, but how could I connect the dots?
I decided it was time to think again in terms of product, something I would love to build and feel passionate about.
Figure 2. Tech demo showcasing my very own ankle.
Rendering gigabytes of voxels in a browser
A CT scan is a 3D grid of density samples, often hundreds of slices deep, which adds up to gigabytes. You can’t download all of it before drawing anything, and you can’t fit all of it in GPU memory. Yet a radiologist expects to spin and re-window the volume with no lag.
The rendering itself is ray casting through a 3D texture. For each pixel on screen, march a ray into the volume, sample density along the way, map each sample to color and opacity, and composite front to back until the pixel goes opaque.
// simplified: march a ray through the CT volume and accumulate color + opacity
vec4 raymarch(vec3 origin, vec3 dir) {
vec4 acc = vec4(0.0);
for (int i = 0; i < STEPS; i++) {
vec3 p = origin + dir * (float(i) * stepSize);
float density = texture(volume, p).r; // sample one voxel
vec4 s = transfer(density); // density -> color + alpha
acc.rgb += (1.0 - acc.a) * s.a * s.rgb; // front-to-back compositing
acc.a += (1.0 - acc.a) * s.a;
if (acc.a > 0.99) break; // early ray termination
}
return acc;
}Three decisions made it run in real time on WebGL. We split each volume into smaller blocks, so the GPU only ever held the parts in view and could skip the empty space between them. We quantized the data, including the normal vectors used for shading, to shrink what had to live in texture memory. And we loaded progressively, coarse first and detail as you leaned in, so the first frame appeared fast even on a multi-gigabyte study.
Figure 3. OpenView Health, the free viewer we built on the set.health SDK. Slices and the 3D render stay in sync, and you can measure, window, and annotate straight in the browser.
Keeping the data safe by never seeing it
Medical data raises the stakes on security, so we ran end-to-end encryption on the client. A scan never left the device in the clear, which meant we couldn’t read it, and that was the point. The data we never held unencrypted is data we could never leak. That one decision collapsed a large amount of compliance and liability surface into something we could reason about.
Announcing set.health
Healthcare is not a commodity, at least it’s not for me, so building a health product requires a strong ethical filter to discard bad ideas. Once you realize that patients are the true owners of the data, but still they can’t access it easily, you can see there is a problem. People need a diagnosis to feel safe and well-informed, to be able to access and share their data and even having access to a Wifi hotspot is critically important so they can connect with their loved ones in the emergency room.
Today I am announcing set.health, my bottom-up approach to solve this problem by helping companies build better software and health products: patient apps, custom implant companies to ship faster, health platforms to communicate better, securing the medical data, funneling anonymized medical data into truly open datasets… and ultimately serve as a product for patients to understand and share their medical data.
There are quite a few companies that also try to solve this problem under different approaches, and I love it!
Figure 4. Technical demo of a COVID19 case using set.health directly from a browser.
What it became
set.health shipped as an SDK and a compliant backend across iOS, Android, and the web, and it ran until 2026. The use case that clicked was custom prosthetics. A doctor could shape an implant against a patient’s real anatomy, an engineer could validate it, and the two could hold a precise conversation inside the same 3D view, all running on our SDK.
Figure 5. A custom-implant planning and ordering tool a partner built on the SDK: plan plates, screws, and surgical guides against a real fracture, leave notes, and order the printed parts. The tool is no longer online.
2020 is being a complicated year for all of us, but It Gets Worse Before It Gets Better. Today, I can say that breaking my ankle might be one of the 5 best things that happened in my life, just time will say in which position exactly.
Please take a look, and send me an email if you wanna know more, I would love to heard from you!
set.health is no longer online — those links go to how the site looked the week I published this.
Tschüss 👋