TorchVision | PyTorch Developer Day 2020

Key Takeaways

[Music] hi everyone and welcome to the 2020 pie charts developer day i'm francisco masa and i'm a research engineer at facebook ai i work in the pytorch team and i'm the maintainer of torch vision today i'll be talking about what torch vision is how it can enable you to get started in computer vision with pytorch and the latest additions that have been introduced to talk to vision over the last year the first question you might ask yourself is what is storage vision torch vision is a library that is built to facilitate research and experimentation in computer vision it extends by torch with functionality which is very specific and useful to computer vision so that keeps the pathwatch library lean and focused on core diploma abstractions but what exactly is their interactivity let's have a closer look trotvision facilitates research and experimentation in computer vision by providing a number of building blocks that allows you to quickly start a new project end-to-end it provides data sets common data sets for a number of mainstream vision tasks such as classification and object detection it also has models we provide reference pre-trained models for classification detection and more protovision also has start scriptable operators specific to computer vision models and data transformation operators to simplify creating data augmentation pipelines additionally it contains primitives for efficient image and video reading and finally reference training scripts which illustrates with a few files how it can train yourself the modern import vision leveraging the building blocks that we provide over the last year we have been improving through division in order to simplify the path from research to production the way pi torch enables the research production path is via throat script an intermediate representation of your python program that it can be exportable to run on c plus environments via the throat script runtime the core part of every deep learning pipeline resides on its models by closely working with the torch script and onyx teams we have enabled all touch vision models to natively support thoughtscript and onixport thanks to the pre-trained models in trout fusion you can quickly build c classifications leveraging the power of deep learning but models are only part of the story when it comes to embedding your python code into a c plus plus runtime very often you need to prepare the input data into a format that is compatible with the model torque division used to rely on pillow for most of the data transformations so moving your application to c plus plus requires implementing the transformations in a way which is compatible with c plus plus now portuguese transforms can directly work controlled answers and can be exported to torch script so you only need to implement the transforms once in python finally tortivision now provides native io functions for image and video decoding enabling the full raw data to model prediction pipeline to be converted from python to c plus plus now let's get a closer look at each one of those points all part division models have native throat script and onix support so that exporting them to c plus plus can be done with a couple of lines of code additionally mixed precision training and inference is available for all models enabling faster runtime with lower memory footprint start vision transforms have been updated to inherit from any module and accept storage sensors as input with torch crypt compatibility this brings several benefits to the data augmentation pipeline such as off-the-shelf gpu support video use case enablement via efficient batch transformations of frames and the possibility to explore the transforms together with your model in c plus plus environments tortivision now provides image reading and writing operators for jpeg and png formats with native transcript support you can directly read local image path into a torch dancer or alternatively decompose the operators into raw file reading where only the bytes of the file are read and returned as the uint8 tensor and the coating of the raw data into an image sensor thanks to native storage script support image decoding can be embedded together with your model for an end-to-end export experience finally trotvision provides a frame based video written api with support for both audio and video and short scripts compatibility the video reader is an iterator and can be combined with eater tools for advanced video reading patterns such as written the next 10 frames after a specified timestamp or skipping every other frame in the video or reading all frames between two timestamps this new video reader api is currently released as beta and its api can change following user feedback we have some additional examples of the new functionalities in short vision in the examples folder on github where you can find notebooks for the video reader api as well as the transforms in torque vision we welcome any contributions if you find a bug have a request for an improvement or a new feature inclusion let us know via the issue tracker on the torch vision github page or contribute via pull requests i'd like to thank you for watching and i hope that your vision can help you build great new applications leveraging [Music] pytorch