Nvidia Jetson Orin Nano Super, PaliGemma 2, Segment Anything Model (SAM) 2.1
Vision Geek Newsletter #10
Hello there,
Welcome to the next edition of Vision Geek Newsletter, covering the essential news in Computer Vision and Machine Learning, read by 1000+ Computer Vision practitioners.
Let’s get into it.
Nvidia Jetson Orin Nano Super
Nvidia has announced a “super” update to their existing edge device “Jetson Orin Nano” and renaming the updated device to “Jetson Orin Nano Super” with a price drop from $499 to just $249 (8GB variant), claiming to be “The World’s Most Affordable Generative AI Computer”.
There is no hardware update. But through a pure software update, they have introduced a new 25W power mode to the device which has increased the performance in all aspects.
1.7x higher generative AI model performance.
67 Sparse TOPs, a significant increase from the previous 40 Sparse TOPs
102 GB/s of memory bandwidth, a significant leap from the previous 65 GB/s memory bandwidth.
1.7 GHz of CPU clock speed, up from 1.5 GHz.
1020 MHz of GPU clock speed, up from 635 MHz.
This compact yet powerful system can effortlessly handle a wide range of LLMs, VLMs, and Vision Transformers (ViTs), from smaller models to those with up to 8B parameters, such as the Llama-3.1-8B model.
For the given price point and the performance, Orin Nano Super can be a great choice as a baseline edge device to handle the modern day AI workloads. Though it is advertised as Generative AI computer, it can handle all forms of ML models.
PaliGemma 2
Google has released the next update to PaliGemma, open source vision-language model in the Gemma family of models. Like its predecessor, PaliGemma 2 uses the same powerful SigLIP and PaLI-3 for vision, but it upgrades to the latest Gemma 2 for the text decoder part.
What’s New
Scalable performance: Optimize performance for any task with PaliGemma 2's multiple model sizes (3B, 10B, 28B parameters) and resolutions (224px, 448px, 896px), where as the previous version had only one variant (3B).
Long captioning: PaliGemma 2 generates detailed, contextually relevant captions for images, going beyond simple object identification to describe actions, emotions, and the overall narrative of the scene.
Expanding to new horizons: Leading performance on chemical formula recognition, music score recognition, spatial reasoning, and chest X-ray report generation, as detailed in the technical report.
PaliGemma 2 is distributed under the Gemma license, which allows for redistribution, commercial use, fine-tuning and creation of model derivatives. The pre-trained models have been designed for easy fine-tuning on custom datasets for specific tasks. Model files are available on Kaggle and HuggingFace. Get started with HuggingFace Transformers or Keras. (Docs | Notebooks | Demo)
Segment Anything Model (SAM) 2.1
Meta’s Fundamental AI Research (FAIR) team has released an update to SAM 2 featuring updated checkpoints with better accuracy.
What’s New
Additional data augmentation techniques to simulate the presence of visually similar objects and small objects where SAM 2 previously struggled.
Improved occlusion handling capability by training the model on longer sequences of frames and making some tweaks to positional encoding of spatial and object pointer memory (updated paper).
SAM 2 Developer Suite, a package of open source code to make it easier than ever to build with SAM 2. This release includes training code for fine-tuning SAM 2 with your own data.
Front-end and back-end code for the web demo.
Not much of changes in model sizes, no. of model variants and inference speed.