This is a question from Chat GPT and Sebastain's book on large language models from scratch. First, this data is interesting. 124 Million parameters GPT_CONFIG_124M = { "vocab_size": 50257, # Vocabulary size "context_length": 1024, # Context length "emb_dim": 768, # Embedding dimension "n_heads": 12, # Number of attention heads "n_layers": 12, # Number of layers "drop_rate": 0.1, # Dropout rate "qkv_bias": False # Query-Key-Value bias } The 1.5 billion parameter GPT model config GPT_CONFIG_1558M = { "vocab_size": 50257, # Vocabulary size "context_length": 1024, # Context length "emb_dim": 1600, # ...
Random AI Thoughts Posts - things I am looking at: scale ai Pinecone Perplexity Notion Hebbia Character AI --- web server web client stand alone - limited html and http - validates your html and shit very strict -- tomcat based -- my own ML-From-Scratch Description: This repository provides implementations of various machine learning algorithms from scratch in Python, without using libraries like Scikit-learn. It's educational and shows how algorithms like k-means clustering, decision trees, and linear regression work under the hood. Repository: ML-From-Scratch on GitHub You-Get Description: A simple and powerful tool for downloading media files from the web. It uses machine learning for analyzing and categorizing the downloaded media. Features: Supports various media sites, custom download options, and automated media organization. 3. OpenAI Gym Description: A toolkit for developing and comparing reinforcement learning algorit...
Comments
Post a Comment