https://medplum.com logo
Join Discord
Powered by
Size Limits for File Uploads
# support
k
Hello! I am using 
medplum.createBinary
to store in a 
DocumentReference
resource. The file sizes can range from small to very large. I am running into an error that says "File too large". I tried finding a size limit in the docs but was unable to find anything. What is the size limit for uploaded files?
s
Also curious about this- any insight @rahul1?
s
this is a restriction we're running into in prod - high priority just tagging for visibility @reshma if there's a cap we could have adjusted with configuration that would be ideal!
r
I'm looking into where this limit may be set. Stay tuned
Hi @kwillard95 - i'm trying to figure out what may be going wrong here. Some clarifying questions: 1. Are you self-hosting, or on our hosted instance? 2. Are you passing the binary as a 
File
, 
Buffer
, or 
UintArray8
3. What are you passing as your content type ? 4. Do you have a URL from which you are downloading these big files?
k
@rahul1 Thanks for your response! 1. We are on Medplum's hosted instance 2. We are passing the binary using 
medplumcreateBinary
and data is 
UintArray8
3. Content type is 
application/pdf
4. No, we are receiving the file directly from the user and then converting into the proper format and data type and then passing that raw file data through to the 
DocumentReference
r
Ok - looking through the code I don't see an obvious place where you sould be limited. Would you be able to post an example call that you're making (without sensitive data)
Would also love to understand what file size you are getting failures at
s
its a 12MB file
looking into getting an example call!
r
Great - I will try to repro on my end. 12 MB should be no big deal. Just to confirm, have you been able to upload via the Medplum app?
k
Yes I just tried to upload via the app and was able to!
r
k
@rahul1 I think this may be an issue with a library we are using to convert jpegs and pngs to pdfs that is causing this error
r
So this is the code that our upload React component uses. It also uses 
createBinary
, so there shouldn't be a major difference between the API calss
@kwillard95 - does this mean the issue is in the browser, not the server?
@kwillard95 - just wanted to check in since I know this is affecting prod ; are you still having issues with the medplum client?
k
@rahul1 Yes, I think we have narrowed it down to the medplum server where we are getting this error from the code here: https://github.com/medplum/medplum/blob/d00839a28519a8d4721bc4c809478b25c1fd1f04/packages/server/src/app.ts#L110. We are having a hard time getting the request call but we are using 
medplum.graphql
r
@kwillard95 - got it. A copy of the raw POST request might be helpful here
if you are using graphQL, does that mean you are not using the 
medplum.createBinary
SDK function?
or is it a combination
s
its a combination
step one is createBinary - that succeeds with 201 created
the next step is a graphql POST requeust - that one is 400ing with that failure "file too big"
and we don't actually get any feedback in the browser about the state of that request, it gets clobbered
r
Would you mind sharing the GraphQL request? Most likely, it's same issue that they were running into here (https://discord.com/channels/905144809105260605/1168641316071157852), where the JSON payload itself is too large
With other users, what I've seen is that they sometimes put the full binary data string in the POST request, which can blow over the max limit (i believe it is 10 MB)?
s
ok that would actually track - so what is the solution if that is the root problem?
do we just need to not include the data string and only include the reference?
r
Yes. And you might want to consider using the 
createAttachment
SDK helper
Have you read the guide on Binary data?
> do we just need to not include the data string and only include the reference? The Binary resource is handled specially by the server, and stores the large file contents in S3. After that, all the other resources that embed that data (e.g. DocumentReference, Media) just reference that blob, typically via the 
Attachment
datatyope
s
I see, yes we just confirmed this is the root cause - so our display logic was depending on data being actually contained within the document reference
but if im understanding correctly we need to only upload the document via create binary, and then reference that binary in the doc reference, and then change our display behavior to traverse from the doc reference to the binary and display it from the binary
r
What HTML element are you using to display?
> but if im understanding correctly we need to only upload the document via create binary, and then reference that binary in the doc reference, and then change our display behavior to traverse from the doc reference to the binary and display it from the binary That is correct! That's the pattern we illustrate here https://www.medplum.com/docs/charting/external-documents#example-creating-a-documentreference-for-a-binary-file
s
these are pdf's we're using a pdf display library called pdfkit / pdf-lib
awesome, thanks @rahul1 ! I think we've got it
r
For display, see our instructions on how we use pre-signed URLs: https://www.medplum.com/docs/fhir-datastore/binary-data#consuming-a-fhir-binary-in-an-application
This gives you URL that you can direcly use in an 
<a>
or 
<object>
tag
s
perfect, yep! we'll use this - shouldn't be a big change
k
@rahul1 Upon making the changes mentioned above, I ran into an issue using the Binary data with react-pdf (the library we are using to view PDF's from within the app). The Binary reference seems to be both a url to`storage.medplum` while also being a Binary string. The PDFViewer can handle urls and 
UInt8Array
but when I pass it as a url, it turns into a binary string and when I try to transform into 
UInt8Array
, there is an error saying the pdf is invalid. How do I transform this Binary data into a 
UInt8Array
or 
URL
so that it is consumable by an external library?
r
@kwillard95 - are you having trouble able to use the 
storage.medplum.com
string as a URL? That would be the recommended way
@kwillard95 - i deleted the message, in case that link was to a PDF with sentive data
One quick test - Are you able to put the 
storage.medplum.com/....
into your browser?
k
Sorry @rahul1 , I figured out the stuff from before but now I've run into something else. I'm using 
medplum.download
to download the Binary as a blob, but I am having a trouble using the direct URL to get the blob. Instead, I am parsing the URL string to build a Binary reference but I know this isn't the best way to go about it. When I pass the URL, I get a "Can't fetch" error. Am I using this method wrong?
r
IS the 
Binary
referenced from a DocumentReference?
k
Yep, the 
Binary
url is referenced from a DocumentReference
I'm getting a CORS error with the 
storage.medplum
url while trying to fetch
r
Oh ok @kwillard95 , thanks for the detail. I'll try to get to the bottom of that today and get back to you
Do you have an example URL with non-sensitive data that you could DM me? It could help me debug
k
r
Thanks. Do you actually mind re-posting this latest issue (CORS) as a new post? Helps for visibility for other folks who may have the same problem. I'll dig in
129 Views
PreviousNext
Powered by