2024 · 10 citations · 3 references
ComputingComputer ScienceNatural Language ProcessingSoftware EngineeringUser Experience DesignCode GenerationEnd-user ProgrammingNatural Language GenerationSoftware TestingText GenerationUsability TestingAutomated Hints GenerationCost-benefit Trade-off AnalysisPrompt-based GenerationUser Study EvaluationUser-guided Generation
We introduce a novel workflow, TiCoder, designed to enhance the trust and accuracy of LLM-based code generation through interactive and guided intent formalization. TiCoder partially formalizes ambiguous intent in natural language prompts by generating a set of tests to distinguish common divergent behaviours in generated code suggestions. We evaluate the code generation accuracy improvements provided by TiCoder at scale across four competitive LLMs, and evaluate the cost-benefit trade off of evaluating tests surfaced by TiCoder through a user study with 15 participants.
3
Evaluating Large Language Models Trained on Code
arXiv (Cornell University) · 2021 · 1.4K citations · Full text
A theory of formal synthesis via inductive learning
Susmit Jha, Sanjit A. Seshia · Acta Informatica · 2017 · 99 citations