Banerjee's picture

1 3

Banerjee

port8080

·

port8080

AI & ML interests

datasets

Recent Activity

upvoted an article about 5 hours ago

FastRTC: The Real-Time Communication Library for Python

reacted to jsulz's post with 👍 3 months ago

Doing a lot of benchmarking and visualization work, which means I'm always searching for interesting repos in terms of file types, size, branches, and overall structure. To help, I built a Space https://huggingface.co/spaces/jsulz/repo-info that lets you search for any repo and get back: - Treemap of the repository, color coded by file/directory size - Repo branches and their size - Cumulative size of different file types (e.g., the total size of all the safetensors in the repo) And because I'm interested in how this will fit in our work to leverage content-defined chunking for versioning repos on the Hub - https://huggingface.co/blog/from-files-to-chunks - everything has the number of chunks (1 chunk = 64KB) as well as the total size in bytes. Some of the treemaps are pretty cool. Attached are https://huggingface.co/black-forest-labs/FLUX.1-dev and for fun https://huggingface.co/datasets/laion/laion-audio-preview (which has nearly 10k .tar files 🤯)

reacted to jsulz's post with 🔥 3 months ago

Doing a lot of benchmarking and visualization work, which means I'm always searching for interesting repos in terms of file types, size, branches, and overall structure. To help, I built a Space https://huggingface.co/spaces/jsulz/repo-info that lets you search for any repo and get back: - Treemap of the repository, color coded by file/directory size - Repo branches and their size - Cumulative size of different file types (e.g., the total size of all the safetensors in the repo) And because I'm interested in how this will fit in our work to leverage content-defined chunking for versioning repos on the Hub - https://huggingface.co/blog/from-files-to-chunks - everything has the number of chunks (1 chunk = 64KB) as well as the total size in bytes. Some of the treemaps are pretty cool. Attached are https://huggingface.co/black-forest-labs/FLUX.1-dev and for fun https://huggingface.co/datasets/laion/laion-audio-preview (which has nearly 10k .tar files 🤯)

View all activity

Organizations

port8080's activity

New activity in xet-team/lfs-analysis 5 months ago

LFS Analysis Roadmap

#3 opened 5 months ago by