Dataset viewer documentation

cuDF

Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

cuDF

cuDF is a Python GPU DataFrame library.

To read from a single Parquet file, use the read_parquet function to read it into a DataFrame:

import cudf

df = (
    cudf.read_parquet("https://huggingface.co/datasets/tasksource/blog_authorship_corpus/resolve/refs%2Fconvert%2Fparquet/default/train/0000.parquet")
    .groupby('sign')['text']
    .apply(lambda x: x.str.len().mean())
    .sort_values(ascending=False)
    .head(5)
)

To read multiple Parquet files - for example, if the dataset is sharded - you’ll need to use dask-cudf:

import dask
import dask.dataframe as dd

dask.config.set({"dataframe.backend": "cudf"})

df = (
    dd.read_parquet("https://huggingface.co/datasets/tasksource/blog_authorship_corpus/resolve/refs%2Fconvert%2Fparquet/default/train/*.parquet")
)
< > Update on GitHub