Skip to main content

Building Models & Improvement Wishes -Llama Stack RFC

Building Models -Llama Stack RFC

Meta - wants to enable everyone to get the most out of the 405B,

such as 

  • Real-time and batch inference
  • Supervised fine-tuning
  • Evaluation of your model for your specific application
  • Continual pre-training
  • Retrieval-Augmented Generation (RAG)
  • Function calling
  • Synthetic data generation

RFC-0001 - Llama Stack · Issue #6 · meta-llama/llama-toolchain (github.com)

Model Improvement Wishes

Model Capabilities

  • Structured Output: Improved support for generating structured data (e.g., JSON) through restricted prediction.
  • Knowledge Distillation: Efficient tools and pipelines for transferring knowledge from larger models to smaller ones.

Fine-Tuning

  • Continued Pre-training: Easier access to sampled pre-training data for maintaining data distribution consistency.
  • Preference Optimization: Best practices and recipes for fine-tuning using Rejected Sampling (RS) and Direct Preference Optimization (DPO).

Model Architecture

  • Agentic Capabilities: Tools and interfaces for integrating Monte Carlo Tree Search (MCTS) with LLMs to enhance logical reasoning.

Disciussion

Models

  • Restricted Prediction: It would be awesome to have native and efficient support for restricted prediction along with a pre-defined schema, such as JSON schema support in llama.cpp grammars. This could boost the usability of the models for generating reliable structured data.

Fine-tuning

  • Continued Pretraining: Maintaining the original data distribution during continued pre-training is tricky. A usual approach is to mix sampled pre-training data with new training data. If Llama models could provide access to a sampled pre-training dataset, it would make this process a lot smoother and ensure consistency.

  • Knowledge Distillation: The 405B model is amazing. I wish that we could have an E2E knowledge distillation tool/pipeline/API to fine-tune using token distributions from teacher models.

  • RS/DPO: It is good to know that Llama3.1 has switched from RLFH to Rejected Sampling (RS) and Direct Preference Optimization (DPO) for optimizing preferences. It would be amazing to offer sample fine-tuning recipes or best practices that could help us fine-tune Llama3.1 models effectively and avoid overfitting.

Agentic

  • MCTS + LLM: There have been some cool attempts to use Monte Carlo Tree Search (MCTS) to boost the logical reasoning capabilities of LLMs. However, there’s still a gap when it comes to tools or viable paths for tightly integrating MCTS with LLMs, except for some rumored projects in closed-source models. Creating a robust tool or interface for this integration would be a huge win for AI agent developers.

Comments

Post a Comment

Popular posts from this blog

Telecom OSS and BSS: A Comprehensive Guide

  Telecom OSS and BSS: A Comprehensive Guide Table of Contents Part I: Foundations of Telecom Operations Chapter 1: Introduction to Telecommunications Networks A Brief History of Telecommunications Network Architectures: From PSTN to 5G Key Network Elements and Protocols Chapter 2: Understanding OSS and BSS Defining OSS and BSS The Role of OSS in Network Management The Role of BSS in Business Operations The Interdependence of OSS and BSS Chapter 3: The Telecom Business Landscape Service Providers and Their Business Models The Evolving Customer Experience Regulatory and Compliance Considerations The Impact of Digital Transformation Part II: Operations Support Systems (OSS) Chapter 4: Network Inventory Management (NIM) The Importance of Accurate Inventory NIM Systems and Their Functionality Data Modeling and Management Automation and Reconciliation Chapter 5: Fault Management (FM) Detecting and Isolating Network Faults FM Systems and Alerting Mecha...
1 comment
Read more

"Depth-Guard" – 3D Spatial Occupancy monitor Challenge -2

  Project Title: "Depth-Guard" – 3D Spatial Occupancy Monitor 1. The Problem In a smart warehouse, a robot needs to know if a loading zone is clear or occupied. A 2D camera alone can’t tell the difference between a "flat picture of a box" on the floor and an "actual 3D box." The Goal: Build a Python-based system that uses Computer Vision and Depth Perception (AI 3D) to identify objects and determine their 3D volume (Size) and Distance from the camera. 2. Intern Tasks Object Detection: Use a pre-trained model (like YOLOv8) to draw 2D boxes around objects. Depth Mapping: Use a depth estimation model (like MiDaS or a simulated Stereo-depth feed) to calculate how far each object is. Occupancy Logic: If an object is closer than 1 meter and larger than a specific volume, mark the zone as "BLOCKED." Alert System: Print a warning if the 3D space is too crowded. 3. Sample Datasets (Simulation) Since interns may not have 3D cameras (LiDAR/RGB-D), pr...
Post a Comment
Read more

Simple Virtual Waiting Room -Challenge 1

   Simple Virtual Waiting Room (VWR) 1. The Problem Our website can only handle 10 users per minute . If more than 10 people try to access it at once, the server will crash. We need a system that: Counts incoming users. Redirects "overflow" users to a waiting page. Admits them back to the main site one by one as space becomes available. 2. Intern Tasks Create a Gateway: A simple script that checks: if (active_users < 10) { allow } else { send to queue } . Build the Queue: Use a simple list (FIFO) to store user IDs. The Wait Page: A basic HTML page that says: "You are number X in line. Estimated wait: Y minutes." Admission Logic: Every 30 seconds, pull the next user from the queue and "admit" them. 3. Sample Datasets (Simulation) Provide these two datasets to the interns. They should write a script to "read" these files and simulate how their system reacts. Dataset A: The Traffic Surge (Input) This file simulates users arriving at the ...
Post a Comment
Read more