PyTorch Lightning is the new framework for large-scale Deep Learning. Discover the ins and outs of this tool below.
What is PyTorch Lightning?
In order to talk about Lightning, we’ll first introduce Pytorch :
Pytorch is a module that provides tools for creating scalable Deep Learning models and training Neural Networks. It uses computer vision and natural language processing.
Developed by Meta.AI, it is now part of the Linux Foundation and is one of the fastest-growing deep learning software and libraries (including FastAi, Hugging Face).
What makes PyTorch Lightning so special?
PyTorch Lightning has 2 special features:
- Tensors (management of multi-dimensional arrays) that accelerate computation via GPU
- Deep neural networks based on automatic on-band differentiation systems
The main modules and classes used are torch.nn, torch.optim, Dataset and Dataloader. Quantum computing is one of its attractive applications.
On the other hand, PyTorch Lightning is an open source framework that replaces and uses PyTorch when model complexity increases, and dissociates research from engineering for greater operationality. It was adopted in 2019 at the Conference: Workshop on Neural Information Processing Systems.
Used for both the research workflow and the production pipeline, it can be deployed in days instead of months, and avoids a lot of clutter and boilerplate code (code independent of experimentation or training).
It provides a high-level interface for PyTorch and can run and train on a variety of hardware. Models are device-independent, which means they have no compatibility problems.
For example, a mobile application is device- or hardware-independent if it can run on any type of device. Tasks are divided up and solved between several processes or components or computers, each of which has its own memory system, and these devices communicate by transmitting messages.
Some of the difficulties encountered in parallel or distributed solving are that the clocks between the components need to be synchronized.
You can get started for free.
Among other capabilities, PyTorch Lightning has:
- Performance and bottlenecks: the bottleneck that exists when the speed at which your data is accessed cannot meet system requirements;
- Model checkpoint: this is a snapshot of your working model stored in non-volatile memory, essential for recovering a specific model state in the event of a failure;
- 16-bit precision: a form of digital format storage that speeds up processes that do not require high precision;
- Loggings: messages via your code to help you read and debug ;
Metrics; - Early-Stopping: which, in machine learning, is a form of regularization that avoids “overfitting” when training a learner with an iterative method.
Conclusion
PyTorch Lightning is an exceptional framework that lets you work easily on complex Deep Learning problems, with a pleasant interface and a large number of tutorials and documentations.
Linkography
- Wikipedia, PyTorch Lightning : https://en.wikipedia.org/wiki/PyTorch_Lightning
- Techopedia, Hardware-agnostic : https://www.techopedia.com/definition/31685/hardware-agnostic
- Wikipedia, PyTorch : https://en.wikipedia.org/wiki/PyTorch
- CNIPS : https://en.wikipedia.org/wiki/Conference_on_Neural_Information_Processing_Systems
- Ligthning : https://lightning.ai/
- Performance bottleneck : https://docs.oracle.com/cd/E19636-01/819-2326/gaxpt/index.html#:~:text=A%20performance%20bottleneck%20occurs%20when,access%20points%20within%20a%20system
- Checkpointing : https://docs.google.com/document/d/1QPHP2JNt7EKNo4nmwIMSm42oyvhWKOTLDUmlkIme0Ok/edit and https://www.indusmic.com/post/checkpoints-in-deep-learning#:~:text=Checkpoints%20are%20snapshots%20of%20your,up%20from%20where%20they%20left
- https://www.google.com/search?q=Early+stopping&oq=Early+stopping&aqs=chrome..69i57j0i512l9.2052j1j7&sourceid=chrome&ie=UTF-8
- https://www.assemblyai.com/blog/pytorch-lightning-for-dummies/
- http://www.idris.fr/jean-zay/gpu/jean-zay-gpu-lightning-multi.html
- https://docs.wandb.ai/v/fr/integrations/lightning
