Publications

The following publications are ones where I've been a significant contributor. For a complete list of publications, please see my Google Scholar profile.

• Steering Semantic Data Processing with DocWrangler
  Shreya Shankar*, Bhavya Chopra*, Mawil Hasan, Stephen Lee, Björn Hartmann, Joseph M. Hellerstein, Aditya G. Parameswaran, Eugene Wu
  Preprint (Under Review)
• RAG Without the Lag: Interactive Debugging for Retrieval-Augmented Generation Pipelines
  Quentin Romero Lauro*, Shreya Shankar*, Sepanta Zeighami, Aditya G. Parameswaran
  Preprint (Under Review)
• LLM-Powered Proactive Data Systems
  Sepanta Zeighami, Yiming Lin, Shreya Shankar, Aditya G. Parameswaran
  IEEE Data Engineering Bulletin
• PromptEvals: A Dataset of Assertions and Guardrails for Custom Production Large Language Model Pipelines
  Reya Vir*, Shreya Shankar*, Harrison Chase, William Hinthorn, Aditya G. Parameswaran
  To appear at NAACL 2025
• DocETL: Agentic Query Rewriting and Evaluation for Complex Document Processing
  Shreya Shankar, Tristan Chambers, Tarak Shah, Aditya G. Parameswaran, Eugene Wu
  Preprint (Under Revision at VLDB)
• Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences
  Shreya Shankar, J.D. Zamfirescu-Pereira, Björn Hartmann, Aditya G. Parameswaran, Ian Arawjo
  UIST 2024
• SPADE: Synthesizing Data Quality Assertions for Large Language Model Pipelines
  Shreya Shankar, Haotian Li, Parth Asawa, Madelon Hulsebos, Yiming Lin, J.D. Zamfirescu-Pereira, Harrison Chase, Will Fu-Hinthorn, Aditya G. Parameswaran, Eugene Wu
  VLDB 2024
• What We've Learned From a Year of Building with LLMs
  Eugene Yan, Bryan Bischof, Charles Frye, Hamel Husain, Jason Liu, Shreya Shankar
  O'Reilly Radar
• Building Reactive Large Language Model Pipelines with Motion
  Shreya Shankar, Aditya G. Parameswaran
  SIGMOD 2024 (Demo)
• Revisiting Prompt Engineering via Declarative Crowdsourcing
  Aditya G. Parameswaran, Shreya Shankar, Parth Asawa, Naman Jain, Yujie Wang
  CIDR 2023
• Operationalizing Machine Learning: An Interview Study
  Shreya Shankar*, Rolando Garcia*, Joseph M. Hellerstein, Aditya G. Parameswaran
  CSCW 2024
• Towards Observability for Production Machine Learning Pipelines
  Shreya Shankar, Aditya G. Parameswaran
  VLDB 2023
• Bolt-on, Compact, and Rapid Program Slicing for Notebooks
  Shreya Shankar*, Stephen Macke*, Sarah Chasins, Andrew Head, Aditya G. Parameswaran
  VLDB 2023
• Automatic and Precise Data Validation for Machine Learning
  Shreya Shankar, Labib Fawaz, Karl Gyllstrom, Aditya G. Parameswaran
  CIKM 2023
• Rethinking Streaming Machine Learning Evaluation
  Shreya Shankar, Bernease Herman, Aditya G. Parameswaran
  ICLR 2022: Workshop on ML Evaluation Standards