this post was submitted on 23 Jun 2023
19 points (95.2% liked)

Experienced Devs

3978 readers
1 users here now

A community for discussion amongst professional software developers.

Posts should be relevant to those well into their careers.

For those looking to break into the industry, are hustling for their first job, or have just started their career and are looking for advice, check out:

founded 1 year ago
MODERATORS
snowe@programming.dev
jmk1ng@programming.dev
drewsiferr@programming.dev
19
What's the most cost-effective way to serve lots of large files (lemmy.ml)
submitted 1 year ago by omegastick@lemmy.ml to c/experienced_devs@programming.dev
14 comments fedilink hide all child comments
 

I'm looking at starting a service that involves hosting a lot of LLM models, which are often going to be 16GB+ (compressed). I did a bit of searching for cloud storage providers with cheap egress, and the cheapest I could find is $0.01 per GB, which would still be $0.16+ per download.

How do sites like Huggingface or CivitAI do it? Lots of VC funding?

you are viewing a single comment's thread
view the rest of the comments
[–] otl@lemmy.sdf.org 4 points 1 year ago* (last edited 1 year ago)

So I got curious and wondered how HuggingFace hosts their files. It’s AWS CloudFront:

$ curl -LI https://huggingface.co/facebook/musicgen-large/resolve/main/state_dict.bin

HTTP/2 302 
content-type: text/plain; charset=utf-8
content-length: 1198
location: https://cdn-lfs.huggingface.co/repos/81/...
date: Fri, 23 Jun 2023 07:09:39 GMT
x-powered-by: huggingface-moon
x-request-id: Root=1-64954533-1eb79eed4ea500203f6435cb
access-control-allow-origin: https://huggingface.co
vary: Origin, Accept
access-control-expose-headers: X-Repo-Commit,X-Request-Id,X-Error-Code,X-Error-Message,ETag,Link,Accept-Ranges,Content-Range
x-repo-commit: c19300a6b2b62d29b345ae9eb7b163278e65238a
accept-ranges: bytes
x-linked-size: 6514108999
x-linked-etag: "1f0cf17b5e65c5dd8ba71767371c377f174e4ce1db44bc4d6657825769f26ffd"
x-cache: Miss from cloudfront
via: 1.1 65c7d0c3355767ac8658c2122c8280b6.cloudfront.net (CloudFront)
x-amz-cf-pop: SYD1-C1
x-amz-cf-id: XAU8X4yneUeudylCi_9MeAYZmISCr8OHiBcgjAGcBQT-edrBF6wGCA==

HTTP/2 200 
content-type: application/octet-stream
content-length: 6514108999
date: Fri, 23 Jun 2023 07:06:20 GMT
last-modified: Thu, 08 Jun 2023 19:05:02 GMT
etag: "44ef1b51c0cc2200e29fed5cddbf8e27-408"
x-amz-storage-class: INTELLIGENT_TIERING
x-amz-server-side-encryption: AES256
x-amz-version-id: upVN9_QvGmQZMDWfVECqbytHgWjHTz4t
content-disposition: attachment; filename*=UTF-8''state_dict.bin; filename="state_dict.bin";
accept-ranges: bytes
server: AmazonS3
x-cache: Hit from cloudfront
via: 1.1 dff3fc94ddb54b32b708edf2668b23d2.cloudfront.net (CloudFront)
x-amz-cf-pop: SIN52-P1
x-amz-cf-id: JWTqKUDp8bVPD7Tt4DBySj8zbgT8G60sCA5BMC1cYF4vK9nx45f1Iw==
age: 200
vary: Origin