Visitar URL original
adbar (Adrien Barbaresi) · GitHub
Skip to content
View adbar's full-sized avatar
529 followers · 275 following

Achievements

Achievement: Starstruckx4Achievement: Pair Extraordinairex2Achievement: Galaxy BrainAchievement: Pull Sharkx3Achievement: QuickdrawAchievement: Arctic Code Vault Contributor

Achievements

Achievement: Starstruckx4Achievement: Pair Extraordinairex2Achievement: Galaxy BrainAchievement: Pull Sharkx3Achievement: QuickdrawAchievement: Arctic Code Vault Contributor

Block or report adbar

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
adbar/README.md

Adrien Barbaresi

GitHub stars

⚡  Website  ✍  Blog  ☕  Coffee  ❤️  Sponsor

Senior Research Data Engineer based in Berlin, specializing in NLP and AI.

15+ years of building NLP pipelines and large-scale datasets, from German web corpora for research to multilingual data for LLM and Voice AI training. PhD in computational linguistics.

I like to build in the open. My packages include Trafilatura, a web text extraction library with 7K+ stars and 13M monthly downloads, described in an ACL 2021 paper.

It is used across thousands of projects, including by Hugging Face, IBM, Microsoft Research, NVIDIA, the Allen Institute for AI, Stanford and the Internet Archive.

Recent Blog Posts

Most Popular Blog Posts

Tech Stack

Skills: Python, Rust, PyTorch, Docker, Kubernetes, AWS and more

Pinned Loading

  1. trafilatura trafilatura Public

    Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

    Python 7k 484

  2. htmldate htmldate Public

    Fast and robust date extraction from web pages, with Python or on the command-line

    Python 159 40

  3. simplemma simplemma Public

    Simple multilingual lemmatizer for Python, especially useful for speed and efficiency

    Python 219 18

  4. rust-primes rust-primes Public

    Rust code and command-line utility to compute and visualize prime sequences

    Rust 3 1

  5. courlan courlan Public

    Clean, filter and sample URLs to optimize data collection – Python & command-line – Deduplication, spam, content and language filters

    Python 180 16

  6. German-NLP German-NLP Public

    Curated list of open-access/open-source/off-the-shelf resources and tools developed with a particular focus on German

    537 68