Deep Learning Weekly: Issue 336
Deep Learning Weekly: Issue 336
State of Generative AI in the Enterprise 2024, Introducing Query Pipelines, Clover: Closed-Loop Verifiable Code Generation, a paper on Sleeper Agents, and many more!
This week in deep learning, we bring you State of Generative AI in the Enterprise 2024, Introducing Query Pipelines, Clover: Closed-Loop Verifiable Code Generation, and a paper on Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training.
You may also enjoy Introducing Pinecone Serverless, CI/CD for Machine Learning in 2024: Best Practices & Tips, Understanding and Coding Self-Attention, Multi-Head Attention, Cross-Attention, and Causal-Attention in LLMs, a paper on Towards Conversational Diagnostic AI, and more!
As always, happy reading and hacking. If you have something you think should be in next week's issue, find us on Twitter: @dl_weekly.
Until next week!
Industry
State of Generative AI in the Enterprise 2024
Findings from the Deloitte AI Institute's report tracking generative AI trends, business impacts, and challenges throughout 2024.
OpenAI’s custom GPT Store is now open for business
OpenAI launches GPT Store, a platform where users can share their custom chatbots.
AI-powered search engine Perplexity AI, now valued at $520M, raises $73.6M
Perplexity AI announced that it raised $73.6 million in a funding round led by IVP with additional investments from NEA, Databricks Ventures, and more.
Introducing Pinecone Serverless
The Pinecone team announces Pinecone serverless, a completely reinvented vector database that lets you easily build fast and accurate GenAI applications at up to 50x lower cost.
OpenAI announces steps to prevent its generative AI tools from being used to manipulate elections
OpenAI is taking concrete steps to ensure that its GenAI tools won’t be used to manipulate the outcomes of key elections coming up this year.
MLOps & LLMOps
The LlamaIndex team introduces Query Pipelines, a new declarative API that allows you to orchestrate simple-to-advanced query workflows over your data for different use cases.
CI/CD for Machine Learning in 2024: Best Practices & Tips
An article that delves into actionable strategies for designing a robust CI/CD pipeline for machine learning in 2024.
Scale LLM Inference on Amazon SageMaker with Multi-Replica Endpoints
A post on how to use the new Hardware Requirements object to optimize the deployment of Llama on Amazon SageMaker.
Learning
Generating Images from Audio with Machine Learning
A guide that demonstrates how to generate images from audio using pre-trained models.
Clover: Closed-Loop Verifiable Code Generation
A blog post about Closed-Loop Verifiable Code Generation, a paradigm that checks consistencies in code and enforces correctness in AI-generated code.
A comprehensive post on understanding Self-Attention, Multi-Head Attention, Cross-Attention, and Causal-Attention in LLMs.
Finetune LLMs on your own consumer hardware using tools from PyTorch and Hugging Face ecosystem
A tutorial on how to finetune a 7B parameter model on a typical consumer GPU with LoRA, as well as tools from the PyTorch and Hugging Face ecosystem.
Libraries & Code
Perform data science on data that remains in someone else's server.
A transformer-based text-to-audio model created by Suno.
Papers & Publications
Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training
Abstract:
Humans are capable of strategically deceptive behavior: behaving helpfully in most situations, but then behaving very differently in order to pursue alternative objectives when given the opportunity. If an AI system learned such a deceptive strategy, could we detect it and remove it using current state-of-the-art safety training techniques? To study this question, we construct proof-of-concept examples of deceptive behavior in large language models (LLMs). For example, we train models that write secure code when the prompt states that the year is 2023, but insert exploitable code when the stated year is 2024. We find that such backdoor behavior can be made persistent, so that it is not removed by standard safety training techniques, including supervised fine-tuning, reinforcement learning, and adversarial training (eliciting unsafe behavior and then training to remove it). The backdoor behavior is most persistent in the largest models and in models trained to produce chain-of-thought reasoning about deceiving the training process, with the persistence remaining even when the chain-of-thought is distilled away. Furthermore, rather than removing backdoors, we find that adversarial training can teach models to better recognize their backdoor triggers, effectively hiding the unsafe behavior. Our results suggest that, once a model exhibits deceptive behavior, standard techniques could fail to remove such deception and create a false impression of safety.
Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models
Abstract:
The high cost of full-parameter fine-tuning (FFT) of Large Language Models (LLMs) has led to a series of parameter-efficient fine-tuning (PEFT) methods. However, it remains unclear which methods provide the best cost-performance trade-off at different model scales. We introduce Astraios, a suite of 28 instruction-tuned OctoCoder models using 7 tuning methods and 4 model sizes up to 16 billion parameters. Through investigations across 5 tasks and 8 different datasets encompassing both code comprehension and code generation tasks, we find that FFT generally leads to the best downstream performance across all scales, and PEFT methods differ significantly in their efficacy based on the model scale. LoRA usually offers the most favorable trade-off between cost and performance. Further investigation into the effects of these methods on both model robustness and code security reveals that larger models tend to demonstrate reduced robustness and less security. At last, we explore the relationships among updated parameters, cross-entropy loss, and task performance. We find that the tuning effectiveness observed in small models generalizes well to larger models, and the validation loss in instruction tuning can be a reliable indicator of overall downstream performance.
Towards Conversational Diagnostic AI
Abstract:
At the heart of medicine lies the physician-patient dialogue, where skillful history-taking paves the way for accurate diagnosis, effective management, and enduring trust. Artificial Intelligence (AI) systems capable of diagnostic dialogue could increase accessibility, consistency, and quality of care. However, approximating clinicians' expertise is an outstanding grand challenge. Here, we introduce AMIE (Articulate Medical Intelligence Explorer), a Large Language Model (LLM) based AI system optimized for diagnostic dialogue.
AMIE uses a novel self-play based simulated environment with automated feedback mechanisms for scaling learning across diverse disease conditions, specialties, and contexts. We designed a framework for evaluating clinically-meaningful axes of performance including history-taking, diagnostic accuracy, management reasoning, communication skills, and empathy. We compared AMIE's performance to that of primary care physicians (PCPs) in a randomized, double-blind crossover study of text-based consultations with validated patient actors in the style of an Objective Structured Clinical Examination (OSCE). The study included 149 case scenarios from clinical providers in Canada, the UK, and India, 20 PCPs for comparison with AMIE, and evaluations by specialist physicians and patient actors. AMIE demonstrated greater diagnostic accuracy and superior performance on 28 of 32 axes according to specialist physicians and 24 of 26 axes according to patient actors. Our research has several limitations and should be interpreted with appropriate caution. Clinicians were limited to unfamiliar synchronous text-chat which permits large-scale LLM-patient interactions but is not representative of usual clinical practice. While further research is required before AMIE could be translated to real-world settings, the results represent a milestone towards conversational diagnostic AI.
 | A guest post by
|