Rishabh Singh

I am a research scientist in the Google Brain team, where we work on developing new deep learning architectures for learning programs and program analysis. I develop new program synthesis techniques for helping end-users, students, and programmers. Apart from research, I enjoy playing bridge.

I am currently deeply interested and excited about developing neural architectures for program representations and using them for program synthesis, repair, and security. Some of our latest work in this area:

Neural Programming

Learn&Fuzz: Machine Learning for Input Fuzzing. Patrice Godefroid, Hila Peleg, Rishabh Singh

Neuro-Symbolic Program Synthesis. Emilio Parisotto, Abdel-rahman Mohamed, Rishabh Singh, Lihong Li, Dengyong Zhou, Pushmeet Kohli

AP: Artificial Programming. Rishabh Singh, Pushmeet Kohli

RobustFill: Neural Program Learning under Noisy I/O. Jacob Devlin, Jonathan Uesato, Surya Bhupatiraju, Rishabh Singh, Abdel-rahman Mohamed, Pushmeet Kohli

Neuro-Symbolic Program Repair for Correcting Introductory Programming Assignments. Sahil Bhatia, Pushmeet Kohli, Rishabh Singh

Deep API Programmer: Learning to Program with APIs. Surya Bhupatiraju, Rishabh Singh, Abdel-rahman Mohamed, Pushmeet Kohli

Melford: Using Neural Networks to Find Spreadsheet Errors. Rishabh Singh, Benjamin Livshits, and Benjamin Zorn

Automated Correction for Syntax Errors in Programming Assignments using Recurrent Neural Networks. Sahil Bhatia, Rishabh Singh

TerpreT: A Probabilistic Programming Language for Program Induction. Alexander L. Gaunt, Marc Brockschmidt, Rishabh Singh, Nate Kushman, Pushmeet Kohli, Jonathan Taylor, Daniel Tarlow

News

Oct 18	Sygus summary paper to appear in CACM
Feb 18	Sarfgen to appear at PLDI 2018
Feb 18	Joined the Google Brain team!
Jan 18	Dynamic Neural Program Embeddings to appear at ICLR 2018
Jan 18	RL for Neural Program Synthesis to appear at ICLR 2018
Dec 17	Neuro-symbolic Program Repair to appear at ICSE 2018
Sep 17	WebRelate to appear at POPL 2018
Sep 17	SYNGAR to appear at POPL 2018
Sep 17	Neural Program meta-induction to appear at NIPS 2017
Jul 17	Program Synthesis book in NOW
Jul 17	DACE to appear at OOSPLA 2017
Jul 17	Learn&Fuzz to appear at ASE 2017
Jun 17	RobustFill to appear at ICML 2017
Jun 17	NoFAQ to appear at FSE 2017
Feb 17	Neural FlashFill to appear at ICLR 2017
Oct 16	Fidex to appear at OOPSLA 2016
Apr 16	BlinkFill to appear at VLDB 2016
Apr 16	Quantitative Program Repair (with Loris D'Antoni and Roopsha Samanta) to appear at CAV 2016
Mar 16	Serving on the PC of PLDI 2017
Jan 16	Probablistic Semantic Transformations in Excel (with Sumit Gulwani) to appear at POPL 2016
Jan 16	Serving on the PC of POPL 2017
Dec 15	Conversational Programmers: An Industrial Perspective (with Parmit Chilana and Philip Guo) to appear at CHI 2016
Jul 15	George M. Sprowls Award for Best Dissertation in CS, MIT

Credits: Adapted from Emina's cool webpage design

Publications
ICML 2023	Gabriel Orlanski, Kefan Xiao, Xavier Garcia, Jeffrey Hui, Joshua Howland, Jonathan Malmaud, Jacob Austin, Rishabh Singh, Michele Catasta. Measuring the Impact of Programming Language Distribution
Neural Networks 2022	Marko Vasić, Andrija Petrović, Kaiyuan Wang, Mladen Nikolić, Rishabh Singh, Sarfraz Khurshid. MoET: Mixture of Expert Trees and its application to verifiable reinforcement learning
TOPLAS 2022	Kensen Shi, David Bieber, Rishabh Singh Tf-coder: Program synthesis for tensor manipulations
NeurIPS 2021	Shobha Vasudevan, Wenjie (Joe) Jiang, David Bieber, Rishabh Singh, Hamid Shojaei, C. Richard Ho, Charles Sutton. Learning Semantic Representations to Verify Hardware Designs
NOW 2021	Swarat Chaudhuri, Kevin Ellis, Oleksandr Polozov, Rishabh Singh, Armando Solar-Lezama, Yisong Yue Neurosymbolic Programming
ICML 2021	Xinyun Chen, Petros Maniatis, Rishabh Singh, Charles Sutton, Hanjun Dai, Max Lin, Denny Zhou. SpreadsheetCoder: Formula Prediction from Semi-structured Context
ICML 2021	Joey Hong, David Dohan, Rishabh Singh, Charles Sutton, Manzil Zaheer. Latent Programmer: Discrete Latent Codes for Program Synthesis
ICLR 2021	Augustus Odena, Kensen Shi, David Bieber, Rishabh Singh, Charles Sutton, Hanjun Dai. BUSTLE: Bottom-Up Program Synthesis Through Learning-Guided Exploration
ICLR 2021	Subham Sekhar Sahoo, Subhashini Venugopalan, Li Li, Rishabh Singh, Patrick Riley. Scaling Symbolic Methods using Gradients for Neural Model Explanation
NeurIPS 2020	Hanjun Dai, Rishabh Singh, Bo Dai, Charles Sutton, Dale Schuurmans. Learning Discrete Energy-based Models via Auxiliary-variable Local Exploration
ICML 2020	Jiani Huang, Calvin Smith, Osbert Bastani, Rishabh Singh, Aws Albarghouthi, Mayur Naik. Generating Programmatic Referring Expressions via Program Synthesis
ICLR 2020	Vincent J. Hellendoorn, Charles Sutton, Rishabh Singh, Petros Maniatis, David Bieber. Global Relational Models of Source Code
POPL 2020	Shengwei An, Rishabh Singh, Sasa Misailovic, Roopsha Samanta. Augmented Example-based Synthesis using Relational Perturbation Properties
ESOP 2020	Rong Pan, Qinheping Hu, Rishabh Singh, and Loris D’Antoni. Solving Program Sketches with Large Integer Values
NeurIPS 2019	Hanjun Dai, Yujia Li, Chenglong Wang, Rishabh Singh, Po-Sen Huang, Pushmeet Kohli Learning Transferable Graph Exploration
SAS 2019	Qinheping Hu, Roopsha Samanta, Rishabh Singh, Loris D'Antoni Direct Manipulation for Imperative Programs
ICLR 2019	Marko Vasic, Aditya Kanade, Petros Maniatis, David Bieber, Rishabh Singh Neural Program Repair by Jointly Learning to Localize and Repair
ICLR 2019	Richard Shin, Neel Kant, Kavi Gupta, Chris Bender, Brandon Trabucco, Rishabh Singh, Dawn Song Synthetic Datasets for Neural Program Synthesis
CACM 2018	Rajeev Alur, Rishabh Singh, Dana Fisman, Armando Solar-Lezama. Search-based Program Synthesis
NeurIPS 2018	Xin Zhang, Armando Solar-Lezama, Rishabh Singh. Interpreting Neural Network Judgments via Minimal, Stable, and Symbolic Corrections
ICML 2018	Abhinav Verma, Vijayaraghavan Murali, Rishabh Singh, Pushmeet Kohli, and Swarat Chaudhuri. Programmatically Interpretable Reinforcement Learning
NAACL 2018	Po-Sen Huang, Chenglong Wang, Rishabh Singh, Wen-tau Yih, and Xiaodong He. Natural Langauge to Structured Query Generation via Meta-Learning
PLDI 2018	Ke Wang, Rishabh Singh, and Zhendong Su. Search, Align, and Repair: Data-driven Feedback Generation for Introductory Programming Exercises
ICLR 2018	Ke Wang, Zhendong Su, and Rishabh Singh. Dynamic Neural Program Embeddings for Program Repair
ICLR 2018	Rudy Bunel, Matthew Hausknecht, Jacob Devlin, Rishabh Singh, and Pushmeet Kohli. Leveraging Grammar and Reinforcement Learning for Neural Program Synthesis
ICSE 2018	Sahil Bhatia, Pushmeet Kohli, and Rishabh Singh. Neuro-Symbolic Program Repair for Correcting Introductory Programming Assignments
POPL 2018	Jeevana Inala and Rishabh Singh. WebRelate: Integrating Web Data with Spreadsheets using Examples
POPL 2018	Xinyu Wang, Isil Dillig, and Rishabh Singh. Program Synthesis using Abstraction Refinement
NIPS 2017	Jacob Devlin, Rudy Bunel, Rishabh Singh, Matthew Hausknecht, and Pushmeet Kohli. Neural Program Meta-Induction
SNAPL 2017	Rishabh Singh and Pushmeet Kohli. AP: Artificial Programming
NOW 2017	Sumit Gulwani, Oleksandr Polozov, and Rishabh Singh. Program Synthesis
ICML 2017	Jacob Devlin, Jonathan Uesato, Surya Bhupatiraju, Rishabh Singh, Abdel-rahman Mohamed, and Pushmeet Kohli. RobustFill: Neural Program Learning under Noisy I/O
ASE 2017	Patrice Godefroid, Hila Peleg, and Rishabh Singh. Learn&Fuzz: Machine Learning for Input Fuzzings
OOPSLA 2017	Xinyu Wang, Isil Dillig, and Rishabh Singh. Synthesis of Data Completion Scripts using Finite Tree Automata
FSE 2017	Loris D'Antoni, Rishabh Singh, and Michael Vaughn. NoFAQ: Synthesizing Command Repairs from Examples
ICLR 2017	Emilio Parisotto, Abdel-rahman Mohamed, Rishabh Singh, Lihong Li, Dengyong Zhou, and Pushmeet Kohli. Neuro-symbolic Program Synthesis
VLDB 2016	Rishabh Singh. BlinkFill: Semi-supervised Programming By Example for Syntactic String Transformations
OOPSLA 2016	Xinyu Wang, Sumit Gulwani, and Rishabh Singh. FIDEX: Filtering Spreadsheet Data using Examples
CAV 2016	Loris D'Antoni, Roopsha Samanta, and Rishabh Singh. Qlose: Program Repair with Quantiative Objectives
CHI 2016	Parmit K. Chilana, Rishabh Singh, and Philip J. Guo. Understanding Conversational Programmers: A Perspective from the Software Industry
POPL 2016	Rishabh Singh and Sumit Gulwani. Transforming Spreadsheet Data Types using Examples
UIST 2015	Mikael Mayer, Gustavo Soares, Maxim Grechkin, Vu Le, Mark Marron, Oleksandr Polozov, Rishabh Singh, Benjamin Zorn, and Sumit Gulwani. User Interaction Models for Disambiguation in Programming by Example
CAV 2015	Rishabh Singh and Sumit Gulwani. Predicting a Correct Program in Programming by Example
SYNT 2015	Rajeev Alur, Dana Fisman, Rishabh Singh, and Armando Solar-Lezama. Results and Analysis of SyGuS-Comp'15
TOCHI 2015	Elena L. Glassman, Jeremy Scott, Rishabh Singh, Philip J. Guo, and Robert C. Miller. OverCode: Visualizing Variation in Student Solutions to Programming Problems at Scale
VMCAI 2014	Rohit Singh, Rishabh Singh, Zhilei Xu, Rebecca Krosnick, and Armando Solar-Lezama. Modular Synthesis of Sketches Using Models
L@S 2014	Elena L. Glassman, Rishabh Singh, and Robert C. Miller. Feature engineering for clustering student solutions
PLDI 2013	Rishabh Singh, Sumit Gulwani, and Armando Solar-Lezama. Automated feedback generation for introductory programming assignments
FMCAD 2013	Rajeev Alur, Rastislav Bodík, Garvit Juniwal, Milo M. K. Martin, Mukund Raghothaman, Sanjit A. Seshia, Rishabh Singh, Armando Solar-Lezama, Emina Torlak, and Abhishek Udupa. Syntax-guided synthesis
VLDB 2012	Rishabh Singh and Sumit Gulwani. Learning Semantic String Transformations from Examples
CACM 2012	Sumit Gulwani, William R. Harris, and Rishabh Singh. Spreadsheet data manipulation using examples
CAV 2012	Rishabh Singh and Sumit Gulwani. Synthesizing Number Transformations from Input-Output Examples
CAV 2012	Rishabh Singh and Armando Solar-Lezama. SPT: Storyboard Programming Tool
FSE 2011	Rishabh Singh and Armando Solar-Lezama. Synthesizing data structure manipulations from storyboards
CAV 2010	Rishabh Singh, Dimitra Giannakopoulou, and Corina Pasareanu. Learning Component Interfaces with May and Must Abstractions
SPIN 2009	Andrey Rybalchenko and Rishabh Singh. Subsumer-First: Steering Symbolic Reachability Analysis
ICSE 2009	Derek Rayside, Zev Benjamin, Rishabh Singh, Joseph Near, Aleksandar Milicevic, and Daniel Jackson. Equality and Hashing for (almost) Free: Generating Implementations from Abstraction Functions

Awards
2015	George M. Sprowls Award for best PhD thesis in Computer Science, MIT
2012-2014	Microsoft Research PhD Fellowship , Microsoft Research
2012	CACM Research Highlight Paper for FlashFill, CACM
2010	William A. Martin Memorial Thesis Award for outstanding Master’s thesis in Computer Science, MIT
2008	Institute Silver Medal for best academic performance in the Department of Computer Science and Engineering, IIT Kharagpur
2008	Bigyan Sinha Memorial Prize for securing 2nd position in the Institute, IIT Kharagpur
2005	Prime Minister’s guest at Repulic Day Parade, Rajpath New Delhi, for securing 1st position in AISSCE CBSE 2004

Rishabh Singh is a research scientist in the Google Brain team. His research interests span the areas of programming languages, formal methods, and machine learning. His recent work has focused on building neural architectures for program synthesis and induction. Prior to joining Google Brain, he was a researcher at Microsoft Research, Redmond from 2014-2018. He obtained his PhD in Computer Science from MIT in 2014, and was awarded the MIT's George M. Sprowls Award for Best PhD Dissertation in Computer Science. He was a Microsoft Research PhD fellow and winner of MIT's William A. Martin Outstanding Master's thesis Award. He obtained his BTech in Computer Science from IIT Kharagpur in 2008, where he was awarded the Institute Silver Medal and Bigyan Sinha Memorial Award. He was also honored to be Prime Minister's National Guest at Republic Day Parade, New Delhi in 2005.

Download the CV(pdf)

My research aims to democratize programming for end-users and students. Towards this goal, I develop systems using program synthesis techniques to make programming more accessible to end-users, students, and even programmers. Some research projects that I am actively working on include:

Data Wrangling for End-users

These systems help data scientists and end-users perform data wrangling (cleaning, transformation, and integration) tasks easily using input-output examples, without writing complex programs/scripts.

BlinkFill

Semi-supervised learning of data transformations from both input-output examples and the input data. 1000x faster than FlashFill and learns richer transforamtions!

Publications	[VLDB16]
Demo	Live Interactive Version

SemFill

Semantic Data Type (Date, Name, Phone Numbers, Address etc.) Transformation in Excel. Probabilistic Learning to handle noisy and inconsistent data.

Publications	[POPL16]
Demo	Live Interactive Version

FlashFill

Help end-users perform string manipulation tasks in Microsoft Excel using input-output examples.

Publications	[CAV12], [VLDB12], [CACM12], [CAV15]
Press	[MIT News], [CNET], [CNN Money], [Wired], [More]
Demo	Video1, Video2, Video3, Try it out in Excel 2013!

Program Synthesis for Education

AutoGrader

Help beginner programmers with automated feedback about the common errors in their code.

Publications	[PLDI13]
Press	[MIT News], [CSAIL Spotlight]
Demo	Live Interactive Version

Storyboard Programming

Help programmers write data-structure manipulations using box-and-arrows diagrams of input-output state configurations.

Publications	[FSE11], [CAV12]
Software	Source Code
Demo	Video