Immanuel Bayer- ipyannotator - the infinitely hackable annotation framework | JupyterCon 2020

Brief Summary In this talk I'm going to explain why standard annotation tools didn't work for us in multiple projects, demonstrate our multiple failed attempts to build a flexible annotation system, and show how we finally came up with ipyannotator - the infinitely hackable annotation framework - and why you should use it, too. Outline Even though much less glamorous than developing new machine learning models, the annotation process and the required tooling is often one of the most critical aspects of real world Machine Learning projects. Many breakthroughs in Machine Learning application such as in image classification, text understanding and recommender systems belong to the class of supervised machine learning. These ML methods often require large collections of input-output pairs from which information is learned. An example for an input-output pair is an animal image together with the species label (name) provided by a human annotator. The main challenge in creating ML datasets is the cost of acquiring annotations/targets which is much more expensive than getting the inputs. It's well known that the prediction quality of ML models critically depends on the amount of training samples for learning. The goal of annotation can be framed as generating as much annotations as possible with sufficient quality under a given budget constraint. Why Jupyter-based annotation tools can save you time and money Back end and front end: the full data science team can help with development Annotation can be turned into supervision by pre-labeling with an existing weak models/heuristics Project specific context can be integrated easily Key requirements and building blocks for a flexible annotation framework Easy integration of complex inputs such as video, audio, 3D point clouds Distributive, collaborative annotation between technical and non-technical users Fast prototyping How you can build your own annotation tool with Jupyter, Voilà and a few lines of code Jupyter plus i