Karnik Ram

I am an ELLIS PhD student supervised by Prof. Daniel Cremers at the Technical University of Munich and Prof. Max Welling at the University of Amsterdam. I'm currently interested in physics-based deep learning and its applications in computational chemistry (esp. green applications).

Previously, I was a Research Associate in the Robotics Institute at Carnegie Mellon University where I worked with Prof. Srinivasa Narasimhan on active 3D sensing and Prof. Kris Kitani on indoor navigation.

Before that, I was an MS by Research student at IIIT Hyderabad, where I worked on improving the robustness of visual-inerital odometry algorithms in dynamic environments and other robot perception problems with Prof. K. Madhava Krishna.

Even earlier, I was a care-free undergrad working on fun projects at SSN College of Engineering, Anna University.

Email | CV | Github | Twitter

What's new
  • March 2024: Another old project on robot safety monitoring using programmable lasers is now under review.
  • Feb 2024: Our work on improving the training of differentiable bundle adjustment layers (V2V) was accepted for CVPR '24.
  • June 2023: My MS thesis at IIIT-H received the Ritesh Tiwari Outstanding MS Thesis Award.
  • April 2023: I will be starting as an ELLIS PhD student at TUM in Fall '23.
  • Oct 2022: Our work on assistive indoor navigation at CMU was featured in Meta Connect.
  • June 2022: Our work on using map priors for inertial odometry was accepted for IROS.
  • Oct 2021: I started as a research associate at CMU RI.
  • June 2021: Our work on RP-VIO was accepted for IROS.

RP-VIO: Robust Plane-based Visual-Inertial Odometry for Dynamic Environments
Karnik Ram, Chaitanya Kharyal, Sudarshan Harithas, K. Madhava Krishna
International Conference on Intelligent Robots and Systems (IROS), 2021.

We present a monocular visual-inertial odometry (VIO) system that uses only planar features and their induced homographies, during both initialization and sliding-window estimation, for increased robustness and accuracy in dynamic environments. We evaluate on diverse sequences, including our own highly-dynamic simulated dataset, and show significant improvement over a state-of-the-art monocular VIO algorithm in dynamic environments.

Project page

From Variance to Veracity: Unbundling and Mitigating Gradient Variance in Differentiable Bundle Adjustment Layers
Swaminathan Gurumurthy, Karnik Ram, Bingqing Chen, Zachary Manchester, J Zico Kolter
To appear in CVPR 2024.

Recent work leveraging learning-based optimizers to tightly-couple correspondence estimation with a weighted least squares objective have shown SOTA results for various pose estimation tasks, but are still difficult to train. We identify possible causes for this instability and propose a simple solution which leads to a 2-2.5x training speedup over a baseline visual odometry model we modify.

Coming soon

Robot Safety Monitoring using Programmable Light Curtains
Karnik Ram, Shobhit Aggarwal, Robert Tamburo, Siddharth Ancha, Srinivasa Narasimhan
Under review.

Developed an inexpensive safety monitoring system for industrials robots using programmable light curtains, a recently developed controllable depth sensor. The system enables fence-less human-robot collaboration, is flexible and scales easily to many robots, all without compromising on safety.

Project page

Learnable Spatio-Temporal Map Embeddings for Deep Inertial Localization
Dennis Melamed, Karnik Ram, Vivek Roy, Kris Kitani
International Conference on Intelligent Robots and Systems (IROS), 2022.

We propose a data-driven prior on possible user locations in a map by combining learned spatial map embeddings and temporal odometry embeddings. Our prior learns to encode which map regions are feasible locations for a user more accurately than previous hand-defined methods, and leads to a 49% improvement in inertial-only localization accuracy when used in a particle filter.

Project page

INFER: Intermediate Representations for Future Prediction
Shashank Srikanth, Junaid Ahmed Ansari, Karnik Ram, Sarthak Sharma, J. Krishna Murthy, K. Madhava Krishna
International Conference on Intelligent Robots and Systems (IROS), 2019.

We have developed an autoregressive model to accurately predict future trajectories of traffic participants (vehicles). We demonstrate that using semantics provides a significant boost and allows the model generalize to completely different datasets, collected across several cities, and also across countries where people drive on opposite sides of the road (left-handed vs right-handed driving).

Preprint | Video

CalibNet: Geometrically Supervised Extrinsic Calibration using 3D Spatial Transformer Networks
Ganesh Iyer, Karnik Ram, J. Krishna Murthy, K. Madhava Krishna
International Conference on Intelligent Robots and Systems (IROS), 2018.

We developed a self-supervised deep network, CalibNet, capable of automatically estimating the 6-DoF rigid body transformation between a 3D LiDAR and a 2D camera in real-time. The network alleviates the need for any calibration targets, thereby reducing significant calibration efforts.

Preprint | Video

PathFinder: Designing a Map-less Navigation System for Blind People in Unfamiliar Buildings
Masaki Kuribayashi, Tatsuya Ishihara, Daisuke Sato, Jayakorn Vongkulbhisal, Karnik Ram, Seita Kayukawa, Hironobu Takagi, Shigeo Morishima, Chieko Asakawa
ACM CHI Conference on Human Factors in Computing Systems (CHI), 2023.

We developed a suitcase robot that allows blind people to find their way around unfamiliar buildings, by detecting and conveying information about intersections and signs. We conducted a user study with seven blind participants which showed that the robot improved their ability and confidence in navigating compared to their regular aid.

Paper | Presentation

Selected projects

Smartphone-based Indoor Navigation
Vivek Roy, Karnik Ram, Kris Kitani  |  Summer 2022

Developed a turn-by-turn assistive indoor navigation app for iOS that combined three deep models for localization in real-time -- LSTM for bluetooth-based absolute position estimation, LSTM for IMU-based relative position estimation, LSTM + U-Net for encoding floor map information. Data collected using Meta's Project Aria Glasses were used for training the models.

Video demo | Presentation | Meta Connect feature

Automatic Calibration of Sensor Extrinsics
Karnik Ram  |  Summer 2018

An end-to-end application with a graphical user interface for easily calibrating the extrinsics between range and visual sensors was developed during GSoC 2018. Automatic and target-less calibration algorithms based on plane-matching and line-matching were integrated into the app, allowing the calibration to be performed in any generic scene setting without the need for any specific targets.

Code | Video demo | Report

ARTPARK Robotics Challenge
Suraj Bonagiri, Viswanarayanan S, Sreeharsha P, Ashwin Rao, Karnik Ram  |  Summer 2021

Mentored and worked with a team in a national-level competition on a janitorial robot to autonomously navigate and clean a washroom setup. The team was selected for the simulation and on-site rounds out of of 136 teams and finished second overall.

Challenge page

Automated Stock Counting Using a Quadcopter
Karnik Ram, Harish S, Apeksha Avinash  |  Winter 2016

Developed a visual odometry module based on optic flow for the localization of a custom-built quadcopter and incorporated it into the PX4 navigation stack, enabling autonomous indoor navigation. All the computations were performed on-board, on an Odroid XU4. A stock counting module was implemented using ArUco markers.

Report | Code

The SSN App
Karnik Ram, Adithya J, Varun R, Muthu CT  |  Winter 2014

Ideated and developed an Android application to notify students and faculty about important events, announcements and other campus related information like bus routes and dining menus. It has close to two thousand users today and is the official app of SSN.

Code | Store | Appreciation | Press


Built using Hugo and Jon's styling
Profile picture courtesy: Sriram
Last updated: March, 2024