Conteúdos sobre LLM

Deep Learning fundamentals

• Pytorch
  • Learn PyTorch for deep learning in a day. Literally.
  • PyTorch Tutorial (Sebastian Raschka)
• NLP
  • Neural Networks: Zero to Hero - Andrej Karpathy
  • Stanford CS224N: Natural Language Processing with Deep Learning
  • Deep Learning Fundamentals - PyTorch Lightning
  • DeepLearning.AI NLP course
  • NYU Deep Learning SP20
    • Week 6: CNN applications, RNN, and attention (lecture) ~2h
    • Week 12: NLP (lecture + practicum) ~2h
  • MIT 6.S191: Recurrent Neural Networks, Transformers, and Attention ~1h
  • LSTM StatQuest ~21min
  • Attention Mechanism - Introduction to Deep Learning
  • CS324 - Large Language Models
  • Understanding Large Language Models - Sebastian Raschka
  • Hugging Face course
  • ChatGPT by Paulo Finardi (Senior DS Itaú) - Muitos links interessantes!
• NLP (consulta)
  • GPT-4 - How does it work, and how do I build apps with it? - CS50 Tech Talk ~67min
  • Livro Natural Language Processing with Transformers
  • NLP by Aman: Attention, Autoregressive vs. Autoencoder Models, Token Sampling Methods, Transformers, BERT
  • A Survey of Large Language Models (27/04/23
  • What Are Transformer Models and How Do They Work? - Cohere
  • Transformers links by Damien
• Parallel and Distributed Training (consulta)
  • Demystifying Parallel and Distributed Deep Learning ~50min
  • CUDA ~1h20min
  • MPI Reduce and Allreduce
  • PyTorch parallel ~1h
  • DeepSpeed

Prompt Engineering:

• Prompt Engineering vs. Blind Prompting
• Prompt Engineering Course by Andrew Ng and Open AI
• Intro to Prompt engineering by Microsoft
• Prompt Engineering Techniques by Microsoft
• Prompt Engineering Guide
• Guia de Engenharia Prompt
  • Few-Shot vs Zero-Shot prompting

LangChain

• LangChain explained ~3min
• LangChain Crash Course ~15min
• LangChain for Beginners ~13min
• The easiest way to work with large language models ~10min
• The LangChain Cookbook - Beginner Guide To 7 Essential Concepts ~38min
• LangChain Crash Course: Build a AutoGPT app in 25 minutes!

• [Langchain PDF App (GUI)

  Create a ChatGPT For Your PDF in Python](https://www.youtube.com/watch?v=wUAUdEw5oxM) ~40min

• LangChain documentation
  • Models: The various model types and model integrations LangChain supports.
  • Prompts: This includes prompt management, prompt optimization, and prompt serialization.
  • Memory: Memory is the concept of persisting state between calls of a chain/agent. LangChain provides a standard interface for memory, a collection of memory implementations, and examples of chains/agents that use memory.
  • Indexes: Language models are often more powerful when combined with your own text data - this module covers best practices for doing exactly that.
  • Chains: Chains go beyond just a single LLM call, and are sequences of calls (whether to an LLM or a different utility). LangChain provides a standard interface for chains, lots of integrations with other tools, and end-to-end chains for common applications.
  • Agents (estilo plug-in do ChatGPT): Agents involve an LLM making decisions about which Actions to take, taking that Action, seeing an Observation, and repeating that until done. LangChain provides a standard interface for agents, a selection of agents to choose from, and examples of end to end agents.
  • Callbacks: It can be difficult to track all that occurs inside a chain or agent - callbacks help add a level of observability and introspection.
• LlamaIndex with LangChain
• Building a Document-based Question Answering System with LangChain, Pinecone, and LLMs like GPT-4
• FedLLM (concorrente)
• Conversational AI : Understanding the Technology behind Chat-GPT

Finetuning/Training

• Finetuning Large Language Models - Sebastian Raschka
• LLaMA-Adapter
• How to train your own Large Language Models
• How to customize LLMs like ChatGPT with your own data and documents

LLMOps

• Building LLM applications for production by Chip Huyen

Reinforcement Learning from Human Feedback

• RLHF - Chip Huyen
• RLHF by Aman
• Awesome RLHF
• Reinforcement Learning for Language Models

Restringindo o output

• NeMo Guardrails from NVIDIA

Evaluation of LLM

• PandaLM (ReProducible and Automated Language Model Assessment) is a open-source LLM that evaluates responses - LLaMA license
• Linkedin post 1
• Linkedin post 2
• LangChain evaluation with LLMs documentation
• Plug-and-Play Bias Detection - HuggingFace

Large Language Models Research - Weekly Digest:

• Nikita Iserson - Lead Machine Learning Engineer at S&P Global

Outros

• Artigos e notícias
  • Pytorch 2.0 alega ser mais eficiente (19/04/23)
  • InstructGPT - Aligning language models to follow instructions - OpenAI (27/01/22)
  • Segurança da informação: New ways to manage your data in ChatGPT
  • Amazon Science - LLM
  • Chaining together Large Language Models with Large Knowledge Graphs
  • Sebastian Raschka’s LinkedIn post on finetuning
• Estudo
  • ChatGPT and LLaMA by Aman
• Bibliotecas
  • Run LLM natively on phones
  • Toolformer: Language Models Can Teach Themselves to Use Tools
  • OpenAI API