title: Dataset caching policy
comments: Hi @loretoparisi, thanks for reporting.
  
  There is an option to force the redownload of the data files (and thus not using previously download and cached data files): `load_dataset(..., download_mode="force_redownload")`.
  
  Please, let me know if this fixes your problem.
  
  I can confirm you that your dataset loads without any problem for me:
  ```python
  In [2]: ds = load_dataset("loretoparisi/tatoeba-sentences", data_files={"train": "train.csv", "test": "test.csv"}, delimiter="\t", column_names=['label', 'text'])
  
  In [3]: ds
  Out[3]: 
  DatasetDict({
      train: Dataset({
          features: ['label', 'text'],
          num_rows: 8256449
      })
      test: Dataset({
          features: ['label', 'text'],
          num_rows: 2061204
      })
  })
  ```  @albertvillanova thank you, it seems it still does not work using:
  
  ```python
  sentences = load_dataset(
       "loretoparisi/tatoeba-sentences",
       data_files=data_files,
       delimiter='\t', 
       column_names=['label', 'text'],
       download_mode="force_redownload"
  )
  ```
  [This](https://colab.research.google.com/drive/1EA6FWo5pHxU8rPHHRn24NlHqRPiOlPTr?usp=sharing) is my notebook!
  
  The problem is that the download file's revision for `test.csv` is not correctly parsed
  
  ![Schermata 2022-04-27 alle 18 09 41](https://user-images.githubusercontent.com/163333/165563507-0be53eb6-8f61-49b0-b959-306e59281de3.png)
  
  If you download that file `test.csv` from the repo, the line `\\N` is not there anymore (it was there at the first file upload).
  
  My impression is that the Apache Arrow file is still cached - so server side, despite of enabling a forced download. For what I can see I get those two arrow files, but I cannot grep the bad line (`\\N`) since are binary files:
  
  ```
  !ls -l /root/.cache/huggingface/datasets/csv/loretoparisi--tatoeba-sentences-efeff8965c730a2c/0.0.0/433e0ccc46f9880962cc2b12065189766fbb2bee57a221866138fb9203c83519
  !ls -l /root/.cache/huggingface/datasets/csv/loretoparisi--tatoeba-sentences-efeff8965c730a2c/0.0.0/433e0ccc46f9880962cc2b12065189766fbb2bee57a221866138fb9203c83519/csv-test.arrow
  !head /root/.cache/huggingface/datasets/csv/loretoparisi--tatoeba-sentences-efeff8965c730a2c/0.0.0/433e0ccc46f9880962cc2b12065189766fbb2bee57a221866138fb9203c83519/dataset_info.json
  ```
   SOLVED! The problem was the with the file itself, using caching parameter helped indeed.
  Thanks for helping!
body: ## Describe the bug
  I cannot clean cache of my datasets files, despite I have updated the `csv` files on the repository [here](https://huggingface.co/datasets/loretoparisi/tatoeba-sentences). The original file had a line with bad characters, causing the following error
  
  ```
  [/usr/local/lib/python3.7/dist-packages/datasets/features/features.py](https://localhost:8080/#) in str2int(self, values)
      852                 if value not in self._str2int:
      853                     value = str(value).strip()
  --> 854                 output.append(self._str2int[str(value)])
      855             else:
      856                 # No names provided, try to integerize
  
  KeyError: '\\N'
  ```
  
  The file now is cleanup up, but I still get the error. This happens even if I inspect the local cached contents, and cleanup the files locally:
  
  ```python
  from datasets import load_dataset_builder
  dataset_builder = load_dataset_builder("loretoparisi/tatoeba-sentences")
  print(dataset_builder.cache_dir)
  print(dataset_builder.info.features)
  print(dataset_builder.info.splits)
  ```
  
  ```
  Using custom data configuration loretoparisi--tatoeba-sentences-e59b8ad92f1bb8dd
  /root/.cache/huggingface/datasets/csv/loretoparisi--tatoeba-sentences-e59b8ad92f1bb8dd/0.0.0/433e0ccc46f9880962cc2b12065189766fbb2bee57a221866138fb9203c83519
  None
  None
  ```
  
  and removing files located at `/root/.cache/huggingface/datasets/csv/loretoparisi--tatoeba-sentences-*`.
   Is there any remote file caching policy in place? If so, is it possibile to programmatically disable it? 
  Currently it seems that the file `test.csv` on the repo [here](https://huggingface.co/datasets/loretoparisi/tatoeba-sentences/blob/main/test.csv) is cached remotely. In fact I download locally the file from raw link, the file is up-to-date; but If I use it within `datasets` as shown above, it gives to me always the first revision of the file, not the last.
  
  Thank you.
  
  ## Steps to reproduce the bug
  ```python
  from datasets import load_dataset,Features,Value,ClassLabel
  
  class_names = ["cmn","deu","rus","fra","eng","jpn","spa","ita","kor","vie","nld","epo","por","tur","heb","hun","ell","ind","ara","arz","fin","bul","yue","swe","ukr","bel","que","ces","swh","nno","wuu","nob","zsm","est","kat","pol","lat","urd","sqi","isl","fry","afr","ron","fao","san","bre","tat","yid","uig","uzb","srp","qya","dan","pes","slk","eus","cycl","acm","tgl","lvs","kaz","hye","hin","lit","ben","cat","bos","hrv","tha","orv","cha","mon","lzh","scn","gle","mkd","slv","frm","glg","vol","ain","jbo","tok","ina","nds","mal","tlh","roh","ltz","oss","ido","gla","mlt","sco","ast","jav","oci","ile","ota","xal","tel","sjn","nov","khm","tpi","ang","aze","tgk","tuk","chv","hsb","dsb","bod","sme","cym","mri","ksh","kmr","ewe","kab","ber","tpw","udm","lld","pms","lad","grn","mlg","xho","pnb","grc","hat","lao","npi","cor","nah","avk","mar","guj","pan","kir","myv","prg","sux","crs","ckt","bak","zlm","hil","cbk","chr","nav","lkt","enm","arq","lin","abk","pcd","rom","gsw","tam","zul","awa","wln","amh","bar","hbo","mhr","bho","mrj","ckb","osx","pfl","mgm","sna","mah","hau","kan","nog","sin","glv","dng","kal","liv","vro","apc","jdt","fur","che","haw","yor","crh","pdc","ppl","kin","shs","mnw","tet","sah","kum","ngt","nya","pus","hif","mya","moh","wol","tir","ton","lzz","oar","lug","brx","non","mww","hak","nlv","ngu","bua","aym","vec","ibo","tkl","bam","kha","ceb","lou","fuc","smo","gag","lfn","arg","umb","tyv","kjh","oji","cyo","urh","kzj","pam","srd","lmo","swg","mdf","gil","snd","tso","sot","zza","tsn","pau","som","egl","ady","asm","ori","dtp","cho","max","kam","niu","sag","ilo","kaa","fuv","nch","hoc","iba","gbm","sun","war","mvv","pap","ary","kxi","csb","pag","cos","rif","kek","krc","aii","ban","ssw","tvl","mfe","tah","bvy","bcl","hnj","nau","nst","afb","quc","min","tmw","mad","bjn","mai","cjy","got","hsn","gan","tzl","dws","ldn","afh","sgs","krl","vep","rue","tly","mic","ext","izh","sma","jam","cmo","mwl","kpv","koi","bis","ike","run","evn","ryu","mnc","aoz","otk","kas","aln","akl","yua","shy","fkv","gos","fij","thv","zgh","gcf","cay","xmf","tig","div","lij","rap","hrx","cpi","tts","gaa","tmr","iii","ltg","bzt","syc","emx","gom","chg","osp","stq","frr","fro","nys","toi","new","phn","jpa","rel","drt","chn","pli","laa","bal","hdn","hax","mik","ajp","xqa","pal","crk","mni","lut","ayl","ood","sdh","ofs","nus","kiu","diq","qxq","alt","bfz","klj","mus","srn","guc","lim","zea","shi","mnr","bom","sat","szl"]
  features = Features({ 'label': ClassLabel(names=class_names), 'text': Value('string')})
  num_labels = features['label'].num_classes
  data_files = { "train": "train.csv", "test": "test.csv" }
  sentences = load_dataset(
       "loretoparisi/tatoeba-sentences",
       data_files=data_files,
       delimiter='\t', 
       column_names=['label', 'text'],
  )
  # You can make this part faster with num_proc=<some int>
  sentences = sentences.map(lambda ex: {"label" : features["label"].str2int(ex["label"]) if ex["label"] is not None else None}, features=features)
  sentences = sentences.shuffle()
  ```
  
  ## Expected results
  Properly tokenize dataset file `test.csv` without issues.
  
  ## Actual results
  Specify the actual results or traceback.
  ```
  Downloading data files: 100%
  2/2 [00:16<00:00, 7.34s/it]
  Downloading data: 100%
  391M/391M [00:12<00:00, 36.6MB/s]
  Downloading data: 100%
  92.4M/92.4M [00:02<00:00, 40.0MB/s]
  Extracting data files: 100%
  2/2 [00:00<00:00, 47.66it/s]
  Dataset csv downloaded and prepared to /root/.cache/huggingface/datasets/csv/loretoparisi--tatoeba-sentences-efeff8965c730a2c/0.0.0/433e0ccc46f9880962cc2b12065189766fbb2bee57a221866138fb9203c83519. Subsequent calls will reuse this data.
  100%
  2/2 [00:00<00:00, 25.94it/s]
  11%
  942339/8256449 [01:55<13:11, 9245.85ex/s]
  ---------------------------------------------------------------------------
  KeyError                                  Traceback (most recent call last)
  [<ipython-input-3-6a9867fad8d6>](https://localhost:8080/#) in <module>()
       12 )
       13 # You can make this part faster with num_proc=<some int>
  ---> 14 sentences = sentences.map(lambda ex: {"label" : features["label"].str2int(ex["label"]) if ex["label"] is not None else None}, features=features)
       15 sentences = sentences.shuffle()
  
  10 frames
  [/usr/local/lib/python3.7/dist-packages/datasets/features/features.py](https://localhost:8080/#) in str2int(self, values)
      852                 if value not in self._str2int:
      853                     value = str(value).strip()
  --> 854                 output.append(self._str2int[str(value)])
      855             else:
      856                 # No names provided, try to integerize
  
  KeyError: '\\N'
  ```
  
  ## Environment info
  ```
  - `datasets` version: 2.1.0
  - Platform: Linux-5.4.144+-x86_64-with-Ubuntu-18.04-bionic
  - Python version: 3.7.13
  - PyArrow version: 6.0.1
  - Pandas version: 1.3.5
  - ```
  
  
  ```
  - `transformers` version: 4.18.0
  - Platform: Linux-5.4.144+-x86_64-with-Ubuntu-18.04-bionic
  - Python version: 3.7.13
  - Huggingface_hub version: 0.5.1
  - PyTorch version (GPU?): 1.11.0+cu113 (True)
  - Tensorflow version (GPU?): 2.8.0 (True)
  - Flax version (CPU?/GPU?/TPU?): not installed (NA)
  - Jax version: not installed
  - JaxLib version: not installed
  - Using GPU in script?: <fill in>
  - Using distributed or parallel set-up in script?: <fill in>
  - ```
  