HuggingFaceApi

HuggingFaceURL(repo_id, [subfolder], filename;
               repo_type = nothing,
               revision = "main")

Construct the real url with the inputs. repo_type is nothing (model), "dataset", or "space".

cached_download(
    hgfurl :: HuggingFaceURL;
    local_files_only :: Bool = HuggingFaceApi.offline_mode(),
    auth_token :: Union{AbstractString, Nothing} = get_token(),
    cache_dir = HuggingFaceApi.hf_hub_cache(),
)

Find the local cache of given url or do downloading. If local_files_only is set (the default when the HF_HUB_OFFLINE environment variable is set), it will try to find the file from cache, and error out when not found. For downloading from private repo, auth_token need to be set, or do HuggingFaceApi.login() beforehand.

The cache uses the same layout as the Python huggingface_hub library, so files are shared between the two.

See also: HuggingFaceURL, login, hf_hub_cache

dataset_info(repo_id; revision = nothing, token = get_token())

Get information for a specific dataset repo repo_id.

delete_token()

Remove token from the disk.

get_file_metadata(url; timeout = 10, headers = [])

Fetch the commit hash, ETag, download location and size of a file on the Hub without downloading it.

get_token()

Get the user access token, from the HF_TOKEN environment variable or the file saved by login (see get_token_path). Returns nothing if no token is found.

get_token_path()

File where login saves the token: $HF_TOKEN_PATH, or $(hf_home())/token.

hf_home()

Root directory of the Hugging Face files shared with the Python huggingface_hub library: $HF_HOME, or $XDG_CACHE_HOME/huggingface, or ~/.cache/huggingface.

hf_hub_cache()

Directory of the download cache: $HF_HUB_CACHE, or $(hf_home())/hub.

hf_hub_download(
    repo_id :: AbstractString,
    filename :: AbstractString;
    repo_type = nothing,
    revision = "main",
    auth_token :: Union{AbstractString, Nothing} = get_token(),
    local_files_only :: Bool  = HuggingFaceApi.offline_mode(),
    cache :: Bool = true,
)

Construct HuggingFaceURL and do cached_download. If cache is false, download file to the temp dir with name generated by tempname(). If local_files_only is set, cache must set.

add 🤗 autocomplete in REPL

list_cache(; cache_dir = HuggingFaceApi.hf_hub_cache())

Get the list of all the cache, as repo type => repo id => revision => files. A revision is a reference (like "main") when one points to the snapshot, otherwise a commit hash.

list_dataset_files(repo_id; revision = nothing, token = get_token())

List all files in the specific dataset repo repo_id.

list_datasets(token = get_token();
              search = nothing, author = nothing, filter = nothing,
              sort = nothing, direction = nothing, limit = nothing,
              full::Bool = false, cardData::Bool = false)

Get information from all datasets in the Hub. You can specify additional parameters to have more specific results.

• search: Filter based on substrings for repos and their usernames, such as pets or microsoft
• author: Filter datasets by an other or organization, such as huggingface or microsoft
• filter: Filter based on tags, such as task_categories:text-classification or language:en.
• sort: Property to use when sorting, such as downloads, likes, lastModified, createdAt or trendingScore.
• direction: Direction in which to sort. Results are sorted in descending order; the Hub only accepts ascending order (1) for createdAt.
• limit: Limit the number of datasets fetched. Results come in pages of at most 1000; without limit, only the first page is returned.
• full: Whether to fetch most dataset data, such as all tags, the files, etc.
• cardData: Whether to grab the metadata for the dataset as well. Can contain useful information such as the PapersWithCode ID.

list_metrics()

Get information from all metrics in the Hub.

list_model_files(repo_id; revision = nothing, token = get_token())

List all files in the specific model repo repo_id.

list_models(token = get_token();
            search = nothing, author = nothing, filter = nothing,
            sort = nothing, direction = nothing, limit = nothing,
            full::Bool = !isnothing(filter), cardData::Bool = false)

Get information from all models in the Hub. You can specify additional parameters to have more specific results.

• search: Filter based on substrings for repos and their usernames, such as resnet or microsoft
• author: Filter models by an author or organization, such as huggingface or microsoft
• filter: Filter based on tags, such as text-classification or spacy.
• sort: Property to use when sorting, such as downloads, likes, lastModified, createdAt or trendingScore.
• direction: Direction in which to sort. Results are sorted in descending order; the Hub only accepts ascending order (1) for createdAt.
• limit: Limit the number of models fetched. Results come in pages of at most 1000; without limit, only the first page is returned.
• full: Whether to fetch most model data, such as all tags, the files, etc.
• cardData: Whether to grab the metadata for the model as well, such as carbon emissions, metrics, and datasets trained on.

list_space_files(repo_id; revision = nothing, token = get_token())

List all files in the specific space repo repo_id.

list_spaces(token = get_token();
           search = nothing, author = nothing, filter = nothing,
           datasets = nothing, models = nothing, linked::Bool = false,
           sort = nothing, direction = nothing, limit = nothing,
           full::Bool = false)

Get information from all Spaces in the Hub. You can specify additional parameters to have more specific results.

• search: Filter based on substrings for repos and their usernames, such as resnet or microsoft
• author: Filter models by an author or organization, such as huggingface or microsoft
• filter: Filter based on tags, such as text-classification or spacy.
• sort: Property to use when sorting, such as downloads, likes, lastModified, createdAt or trendingScore.
• direction: Direction in which to sort. Results are sorted in descending order; the Hub only accepts ascending order (1) for createdAt.
• limit: Limit the number of Spaces fetched. Results come in pages of at most 1000; without limit, only the first page is returned.
• full: Whether to fetch most Space data, such as all tags, the files, etc.
• datasets: Whether to return Spaces that make use of a dataset. The name of a specific dataset can be passed as a string.
• models: Whether to return Spaces that make use of a model. The name of a specific model can be passed as a string.
• linked: Whether to return Spaces that make use of either a model or a dataset.

login(token = <prompt>)

Validate a user access token and save it to get_token_path() for operations that need authentication. The file is shared with the Python huggingface_hub library, so hf auth login works too.

logout()

Remove the token saved by login.

model_info(repo_id; revision = nothing, token = get_token())

Get information for a specific model repo repo_id.

remove_cache(repo_id::AbstractString; repo_type = nothing, revision = nothing, cache_dir = HuggingFaceApi.hf_hub_cache())

Remove all cached files of a given repo, or only those of revision if it is set.

remove_cache(hgfurl::HuggingFaceURL; cache_dir = HuggingFaceApi.hf_hub_cache())

Remove the cached file of the given url, and its blob if no other revision uses it.

save_token(token)

Write token to the disk for future use.

space_info(repo_id; revision = nothing, token = get_token())

Get information for a specific space repo repo_id.

try_to_load_from_cache(hgfurl::HuggingFaceURL; cache_dir = hf_hub_cache())

Return the path of the cached file for hgfurl, or nothing if it is not in the cache. No request is made.