Datasets

Human Pose Estimation

  1. Human3.6M: Large-scale 3d human motion capture datasets for fitness, close human interactions, and self-contact [Human3.6M, Human Motion]
  2. MS COCO: A large-scale object detection, segmentation, and captioning dataset [Common Objects, Context]
  3. MPII: 2D Human Pose Estimation - New Benchmark and State of the Art Analysis [Human Pose Estimation, Benchmark]
  4. CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmark [Pose Estimation, Crowded Scene]

Object Detection

  1. Objects365: A Large-scale, High-quality Dataset for Object Detection [Object detection, Benchmark]
  2. Open Images Dataset: Unified image classification, object detection, and visual relationship detection at scale [Image classification, Object detection]

Large-scale Multi-modality Model

  1. RefCOCO: Large-scale Multi-modality Models Evaluation Suite [Multi-modality Models, Evaluation]
  2. Molmo: Multimodal Open Language Model [Multimodal Language Model]
  3. Data Commons: A single source for publicly available statistical data [knowledge graph, schema, APIs, open-source project platform]
  4. Wikidata Vector Database: semantic embeddings of Wikidata entities [Wikidata, Vector Database, semantic search, knowledge graph]
  5. LAION Science Dataset Explorer: Interactive visualization and exploration of scientific papers from the Aella open science dataset [scientific papers, semantic embeddings, dimensionality reduction, clustering visualizations]

Physics Simulation

  1. The Well: 15TB of Physics Simulations [Physics Simulations, machine learning datasets]

Robot Learning

  1. AgiBot World: The first open-source dataset that systematically covers key research directions in embodied intelligence [embodied intelligence, real-world environments, contact and force feedback, robot manipulation]
  2. RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulation [Multi-embodiment, Robot Manipulation, VLA]
  3. DROID: A Large-Scale In-the-Wild Robot Manipulation Dataset [Robot Manipulation, Large-Scale, In-The-Wild]
  4. Open X-Embodiment: Robotic Learning Datasets and RT-X Models [Robotic Learning, Skills, Robotic Manipulation]
  5. HUMOTO: A 4D Dataset of Mocap Human Object Interactions [human-object interactions, 3D human poses, artists-made object meshes, 4D]
  6. RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot [Robotic Dataset, Robot Manipulation, Diverse Skills]
  7. GraspFactory: A Large Object-Centric Grasping Dataset [CAD, Grasp dataset, Learning]
  8. VinT-6D: A Large-Scale Object-in-hand Dataset from Vision, Touch and Proprioception [Object-in-hand Dataset, Vision, Touch, Proprioception]
  9. OpenRoboCare: A Multi-Modal Multi-Task Expert Demonstration Dataset for Robot Caregiving [Robot Caregiving, Multi-Modal, Multi-Task, Expert Demonstration]
  10. PHUMA: Physically Reliable Humanoid Locomotion Dataset [Humanoid Locomotion]
  11. TWIST Data: Humanoid Data is Better when Universally Sharable [humanoid loco-manipulation, long-horizon, whole-body, dexterous, egocentric]
  12. BridgeData V2: A Dataset for Robot Learning at Scale [Robot Learning , Scale, Multi-task Learning]
  13. MIME: Multiple Interactions Made Easy [Learning from Demonstration, Kinesthetic data]
  14. ARIO: A New Standard and Unified Dataset for Versatile, General-Purpose Embodied Agents [Embodied Agent, physical World, Versatile]
  15. RoboCOIN: An Open-Sourced Bimanual Robotic Data COllection for INtegrated Manipulation [Bimanual Robotic Data Collection, Integrated Manipulation]
  16. OpenLET: Embodied Intelligence Datasets [Whole-Body Motion Control, Dexterous Manipulation, Arm Manipulation]
  17. LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning [Knowledge Transfer, Lifelong Robot Learning, Benchmark]
  18. OmniSharing DB: PaXini Full-modal Datasets [Full-modal Datasets, Tactile Sensing]
  19. Daimon-Infinity: The world’s largest omni-modal robotics dataset for physical AI, including high-resolution tactile sensing [VTLA, multimodal data, physical AI, high-resolution tactile sensing]
  20. HRDexDB: A Paired Human-Robot Dataset for Cross-Embodiment Dexterous Grasping [Cross-Embodiment, Human-to-Robot Learning, Dexterous Manipulation]

Return to Resources