We have one use case where customer has an wearable sensor with huge amount of data stored in time series database, now they want to expose FHIR APIs so we are thinking to use Medplum as backend where existing app db will be synced to Medplum and use Medplums FHIR framework for giving API access to customers, is this the best way or do you think huge timeseres data can cause issue or slowness?

At present this is not a supported/intended scenario and customers who do support it aggregate on the client and send up an aggregated observation to the server. However, we do have interest in this use case for digital biomarkers and plan to pursue it in the future.

I see, thanks for the revert. So in short, large timeseries data without aggregation might not be good fit for now, and in future it will be rolled out.

Hello, I am definitely interested in this conversation ! Indeed in our use case we intend to store Observations with valueSampledData for a one night recording of a patient (1 value per second for 12hours : ~720 values ). My understanding from this conversation is that this should not be stored like that in Medplum ? What do you suggest in that case ?

The data size is not really the issue here, but rather the searches/calculations for time series aggregation and analysis. This isn't a heavily tested scenario today. More typically we see aggregations being written as observations themselves in the datastore. If for example, you are collecting readings on a mobile device or on the edge, aggregating there and writing the summarized version to the record is the more common use case we see.

Yeah thanks. Yes these are edge devices with continuous monitoring of patient, I am checking aggregation part and if that makes sense, but thanks for the inputs and clarity on what is being tested till now.

@thomabig , sampledValue is definitely directionally correct, and is more performant than saving each 1s value as it's own observation. Like @reshma mentioned, the main issue is how these data are used. Typically, the analysis and display involve aggregation into less granular time buckets ot be useful

Hi @rahul1 , @reshma Thanks for the answers ! Indeed my calculation was pretty bad :p we would have 43200 samples. So my understanding is that if we want to retrieve everything to display a graph, this is not an issue, but if we want to have something more specific allowing us to filter on that samples the sampledData is probably not the solution. Our idea here was to use ElasticSearch next to Medplum to have more control over our search

Yes that's right. Down the road, we would like to have better support for high-frequency time series data, but that won't exist in the near future

Thanks