AI Terminology

Given that AI is a highly technical field, the amount of jargon can be daunting. For your ease of reference, I have compiled a comprehensive list of common terms and their meaning. Use CTRL+F to see if the term you're looking for appears below.

Accuracy:: The percentage of an AI model’s predictions that are correct. It is a crucial metric for evaluating a model’s performance on tasks like document classification or predicting outcomes.
Activation Function:: The algorithm within a node of a neural network that determines the node’s output for a given input. For example, a sigmoid function that maps the input to a value between 0 and 1, mimicking the firing of a biological neuron.
Active Learning:: A machine learning approach where the algorithm selectively queries a data source to label new data points, optimizing the learning process. For instance, an AI contract review tool identifying the most informative clauses to analyze and learn from.
Adversarial Machine Learning:: The study of how machine learning models can be manipulated or fooled by malicious inputs designed to cause misclassifications or errors. Adversarial machine learning is relevant for ensuring the robustness and security of AI systems, particularly in high-stakes applications.
Algorithm:: A well-defined set of instructions a computer follows to solve a problem or complete a task, such as the steps an AI system uses to analyze documents and extract relevant information.
Alpaca 7B:: Developed by researchers at Stanford, Alpaca 7B is a fine-tuned model based on Meta’s LLaMA 7B, utilizing 52K instruction-following demonstrations. It demonstrates capabilities similar to OpenAI’s text-davinci-003, offering a compact and cost-effective alternative for academic research in instruction-following AI models.
Anomaly Detection:: The process of identifying rare or unusual data points that deviate significantly from the norm. AI could flag unusual clauses or terms in contracts that may require further review.
Artificial General Intelligence (AGI):: An advanced, hypothetical form of AI that can perform any intellectual task at a human level or beyond. While AGI does not yet exist, its development could have profound implications for various professions and society as a whole.
Bias:: In AI, bias can refer to: 1) A constant in a node’s activation function, 2) The difference between a model’s predictions and the ground truth in training data, or 3) When an AI model performs poorly for certain subgroups, raising fairness and discrimination concerns.
Big Data:: Extremely large datasets that require distributed processing. AI often relies on big data, such as databases of millions of documents, to train models and extract insights.
Black Box:: An AI system whose internal decision-making processes are opaque or difficult to interpret. Black box models can be highly accurate but may raise concerns about accountability and transparency.
Capsule Networks:: An architecture designed to overcome some limitations of traditional convolutional neural networks (CNNs), particularly in terms of capturing spatial hierarchies between features. Capsules are groups of neurons that represent different aspects of the same entity, allowing the network to maintain more information about the entity’s state.
Chatbot:: An AI-powered interface that engages in real-time conversations with users via text. Chatbots can provide basic guidance, answer FAQs, and assist with various tasks.
Classification:: A supervised machine learning task where the model predicts the category or class of input data. For example, classifying documents as contracts, briefs, or opinions based on their content.
Clustering:: An unsupervised machine learning technique that groups similar data points together based on their inherent patterns or structure. Clustering could be used to identify related cases or contracts without explicit labels.
Computer Vision:: The field of AI focused on enabling computers to interpret and understand visual information from images or videos. Computer vision can extract data from scanned documents, detect signatures, or identify key clauses in contracts.
Continual Learning:: The ability of an AI model to continuously learn and adapt to new information or tasks over time without forgetting previously acquired knowledge. Continual learning is essential for AI models to stay up-to-date with evolving landscapes and incorporate new data.
DALL-E:: A large-scale generative model developed by OpenAI that creates images from textual descriptions. DALL-E is trained on a vast dataset of text-image pairs and uses a transformer-based architecture to generate novel and coherent images that match the given text.
Dataset:: A collection of related data used to train and evaluate machine learning models. AI datasets may include collections of documents, images, or other relevant data.
Decision Tree:: A type of machine learning model that makes predictions by learning a series of hierarchical decision rules from data. Decision trees can provide explainable AI decisions, which is valuable in various contexts.
Deep Fake:: Synthetic media generated by AI to deceive, such as realistic fake images, videos, or audio of individuals. Deep fakes pose challenges for evidence authentication and raise concerns about privacy and consent.
Deep Learning:: A subfield of machine learning that uses multi-layered neural networks to learn hierarchical representations from vast amounts of data. Deep learning powers advanced AI applications like predicting outcomes or answering complex queries.
Digital Sandbox:: A secure, isolated testing environment that mimics real-world conditions, used to develop and evaluate new AI technologies without risking actual systems or data. AI tools may be tested in digital sandboxes before deployment.
Edge AI:: Refers to AI algorithms that are processed locally on a hardware device – such as Internet of Things (IoT) devices. Edge AI is crucial for applications requiring real-time processing and decision-making without the latency associated with data transmission to and from a centralized server.
Explainable AI (XAI):: A set of techniques and approaches aimed at making AI models more transparent, interpretable, and accountable by providing human-understandable explanations for their predictions or decisions. Explainable AI is crucial for building trust and ensuring fairness in AI applications.
Explainability vs. Interpretability:: While “Explainable AI” is included, distinguishing between explainability and interpretability could be beneficial. Interpretability refers to the extent to which a human can understand the cause of a decision, while explainability involves the model providing explanations for its decisions in human-understandable terms.
Federated Learning:: A distributed machine learning approach that enables training models on decentralized data without the need for data sharing. Federated learning could allow AI models to learn from sensitive or confidential data across multiple organizations or jurisdictions while preserving privacy.
Few-Shot Learning:: A paradigm in machine learning where a model can learn a new task from a very limited number of examples, often just a few or even one. Few-shot learning could enable AI models to quickly adapt to new domains or document types with minimal training data.
Foundation Model:: A large, pre-trained AI model that learns general knowledge and skills from diverse data, which can then be fine-tuned for specific downstream tasks. Foundation models, trained on vast corpora, could greatly accelerate the development of specialized AI applications.
Generative AI:: AI systems that can create new content, such as text, images, or audio, by learning patterns and representations from training data. Generative AI has the potential to draft documents, summarize content, or even provide creative arguments, but also raises questions about intellectual property and liability.
GPT (Generative Pre-trained Transformer):: A type of language model that uses the transformer architecture to generate human-like text. GPT models have shown impressive performance on tasks like document review and question answering.
Gradient Descent:: An optimization algorithm used to train machine learning models by iteratively adjusting the model’s parameters to minimize a loss function. Gradient descent helps AI models learn to make accurate predictions from data.
Hallucination:: A phenomenon where an AI model generates outputs that are inconsistent, factually incorrect, or nonsensical, often due to biases or limitations in the training data. Mitigating hallucinations is crucial for ensuring the reliability and trustworthiness of AI systems.
HuggingFace:: A popular open-source platform and community that provides a wide range of pre-trained models, tools, and libraries for natural language processing (NLP), computer vision, and audio processing tasks, making it easier for developers to build and deploy AI applications.
img2img:: A feature in some generative AI models that takes an input image and a target text description, and then modifies the input image to match the target description while preserving the image’s overall structure and content.
Inferencing:: The process of using a trained AI model to make predictions or decisions on new, unseen data. For example, an AI model trained on historical data could infer the likelihood of different outcomes for a new, unseen case.
Intelligent Automation:: The application of AI and machine learning to automate complex tasks that typically require human intelligence. Intelligent automation could streamline processes like document review, due diligence, and data analysis.
Knowledge Graph:: A structured representation of entities, their attributes, and the relationships between them in a graph format. Knowledge graphs can model complex interconnections between concepts, enabling more sophisticated reasoning and analysis by AI systems.
KoboldAI:: An open-source project that provides a user-friendly interface for interacting with AI language models, particularly focused on creative writing and role-playing scenarios. It allows users to generate text based on prompts and adjust various generation parameters.
KoboldCPP:: KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF models. It’s a single self contained distributable from Concedo, that builds off llama.cpp, and adds a versatile Kobold API endpoint, additional format support, Stable Diffusion image generation, backward compatibility, as well as a fancy UI with persistent stories, editing tools, save formats, memory, world info, author’s note, characters, scenarios and everything Kobold and Kobold Lite have to offer.
Silly Tavern:: An open-source front-end platform that provides a web-based interface for collaborative story writing and role-playing using AI language models.
Large Language Model (LLM):: A type of AI model trained on vast amounts of text data, enabling it to understand, generate, and reason with human language at an unprecedented scale. LLMs have shown promising results in AI tasks such as document summarization, question answering, and content analysis.
LangChain:: An open-source framework designed to assist in the development of applications with large language models (LLMs). It provides a standard interface for chains of prompts, allowing developers to focus on creating the application instead of worrying about the specific model implementations.
Learner:: The component of a machine learning system that updates the model’s parameters based on the training data and learning algorithm. The choice of learner can significantly impact an AI model’s performance and efficiency.
Llama 2:: A large language model developed by Meta AI, designed to be efficient and scalable while maintaining high performance across a wide range of natural language tasks. Llama 2 focuses on speeding up training and inference times compared to its predecessor.
Lora:: Refers to LoRA (Low-Rank Adaptation of Large Language Models), a technique for fine-tuning large pre-trained language models like GPT-3 by learning a low-rank decomposition of the model’s weight matrices. This allows for efficient adaptation to new tasks with fewer trainable parameters.
Loss Function:: A mathematical function that quantifies the difference between a model’s predictions and the actual values in the training data. Minimizing the loss function is the key objective of training AI models.
Machine Learning:: A subset of AI that focuses on enabling computers to learn and improve from data without being explicitly programmed. Machine learning is the foundation of modern AI, allowing models to extract insights and make predictions from vast datasets.
Meta-Learning:: A subfield of machine learning focused on learning to learn, i.e., developing algorithms that can quickly adapt to new tasks or environments by leveraging past experiences. Meta-learning could help AI models rapidly generalize to new tasks or domains with limited training data.
Minimum:: The point in the loss function where the model’s predictions most closely match the ground truth in the training data. Finding this minimum is the goal of training AI models.
mov2mov:: An emerging concept in AI referring to the task of transforming an input video (mov) into a target video based on a given set of instructions or desired modifications, while preserving the general content and dynamics of the original video.
Multi-Task Learning:: A branch of machine learning where the model is trained on multiple related tasks at the same time, using a shared representation. This can lead to improved learning efficiency and prediction accuracy for the tasks.
Natural Language Processing (NLP):: A branch of AI focused on enabling computers to understand, interpret, and generate human language. NLP is crucial for AI applications that deal with unstructured text data, such as documents, opinions, and reports.
Neural Network:: A machine learning model loosely inspired by the structure and function of biological neural networks. It consists of interconnected nodes (neurons) organized in layers, which process input data to generate outputs. Neural networks are the building blocks of deep learning models used in AI.
Node (Neuron):: A fundamental unit in a neural network that takes input, applies a transformation (activation function), and passes the output to connected nodes. The collective computation of nodes enables neural networks to learn complex patterns in data.
Parameters (LLMs):: Parameters in Large Language Models (LLMs) are the individual elements that the model adjusts during training to learn from data and make predictions. A model with 70 billion (70B) parameters has 70 billion such elements, enabling it to understand and generate complex language by capturing subtle patterns in text. However, the vast number of parameters also means these models are computationally intensive to train and run. 7b, 13b, and 20b models are more easily accessible on consumer hardware.
Pre-trained Model:: A machine learning model that has been trained on a large, general dataset and can be fine-tuned for specific tasks with less data and computation. Using pre-trained models can accelerate the development of AI applications and improve performance on downstream tasks.
Quantization:: The process of reducing the precision of a model’s parameters to create a quantized model, often using techniques like linear or logarithmic scaling to map the original float32 values to a lower-precision representation. Quantization enables faster inference and lower memory usage, making models more efficient and deployable on resource-constrained devices.
Quantization (AWQ):: AWQ is an advanced technique aimed at optimizing the performance of Large Language Models (LLMs) through efficient and accurate low-bit weight quantization, such as INT3/4. It is specifically designed to support a wide range of LLMs, including instruction-tuned models and multi-modal Language Models (LMs), by enabling them to operate more efficiently, especially in resource-constrained environments.
Quantization (GPTQ):: GPTQ is a novel post-training quantization technique designed for GPT models, which significantly reduces their computational and storage demands without compromising on performance. It efficiently compresses GPT models to 3 or 4 bits per weight, achieving remarkable compression gains and enabling the execution of models with up to 175 billion parameters on a single GPU, with minimal loss in accuracy. This method facilitates greater accessibility and efficiency in using large GPT models, offering substantial inference speedups.
Quantization (GGUF):: GGUF is a quantization method tailored for CPUs and Apple M series devices, enhancing the efficiency of Large Language Models by allowing selective layer execution on GPUs. It’s designed for speed and flexibility, especially on non-GPU hardware, making LLMs more accessible and manageable.
Quantization (GGML):: GGML, created by Georgi Gerganov, is a C library that facilitates machine learning by defining low-level ML primitives and a binary format for LLM distribution. It employs quantization to enable running LLMs on consumer hardware, democratizing access to advanced AI technologies through efficient model compression.
Quantum Machine Learning:: An emerging field at the intersection of quantum computing and machine learning. Quantum machine learning explores how quantum algorithms can improve machine learning tasks, potentially offering faster processing and novel ways of handling data.
Random Forest:: An ensemble machine learning method that combines multiple decision trees to make robust predictions. Random forests can improve the accuracy and stability of AI models, particularly for tasks like predicting outcomes or identifying relevant documents.
Reinforcement Learning:: A type of machine learning where an agent learns to make decisions by performing actions in an environment to achieve a goal. The agent learns from feedback in the form of rewards or penalties.
Sandbox:: See Digital Sandbox.
Sampling Method:: In the context of generative models like Stable Diffusion, a sampling method refers to the algorithm used to generate new data points from the learned probability distribution. Examples include DPM++ 2M SDE Karras, Eular, and DDIM sampling methods, which differ in their trade-offs between generation quality and speed.
Self-Supervised Learning:: A learning paradigm where the model learns to predict part of the input from other parts of the input, using a pretext task. It is a form of unsupervised learning that can leverage large amounts of unlabeled data effectively.
Sentiment Analysis:: The process of using NLP to determine the emotional tone or opinion expressed in a piece of text. Sentiment analysis can be used to gauge public opinion on issues, analyze testimonies, or assess the tone of documents.
Sharding:: The process of horizontally partitioning a large dataset across multiple distributed servers or machines. Sharding enables the storage and processing of massive datasets that exceed the capacity of a single machine, facilitating the training of large-scale AI models.
Stable Diffusion:: A deep learning, text-to-image model developed by Stability AI that generates detailed images conditioned on text descriptions, using a diffusion-based modeling process. It is known for its ability to produce high-quality, diverse images with coherent styles and compositions.
Stable Diffusion Checkpoint:: A snapshot of the state of a trained Stable Diffusion model at a particular point during the training process. Checkpoints allow users to resume training from a specific point or use the model for inference with the learned parameters up to that point.
Structured Data:: Data that is organized in a predefined format, such as a table with rows and columns. Examples of structured data include metadata, clauses, and entity information.
Temperature:: In the context of Large Language Models (LLMs), “temperature” is a parameter that controls the randomness of predictions by scaling the output probabilities. A low temperature results in less randomness, making the model’s outputs more deterministic and often higher quality. Conversely, a high temperature increases randomness, leading to more varied outputs. Adjusting the temperature allows for balancing between creativity and coherence in the model’s generated text.
Tensor:: A multi-dimensional array used to represent and manipulate data in deep learning frameworks. Tensors are the fundamental data structures used to store and process model parameters, inputs, and outputs.
Tensor Library:: A software library that provides efficient implementations of tensor operations and mathematical functions, optimized for performance on various hardware platforms (e.g., CPUs, GPUs, TPUs). Examples include PyTorch, TensorFlow, and MXNet.
Training Data:: The dataset used to train a machine learning model, typically consisting of input-output pairs. Training data may include labeled examples of documents, outcomes, or expert annotations.
Transfer Learning:: A machine learning technique where knowledge gained from solving one task is applied to a related task, enabling faster learning and improved performance with less data. Transfer learning can help adapt general AI models to specific domains or use cases.
Transformer:: A neural network architecture designed to process sequential data, such as text, by learning contextual relationships between elements. Transformers have revolutionized NLP and form the basis of state-of-the-art language models used in AI.
text-generation-webui:: The text-generation-webui is a highly popular web interface known for its extensive range of features and potent add-ons. It also enables GPU acceleration for enhanced performance.
txt2img:: A common functionality in generative AI models that takes a textual description as input and generates a corresponding image as output, leveraging the model’s learned associations between text and visual features.
Unstructured Data:: Data that does not conform to a predefined structure or format, such as free-form text, images, or audio. Documents like contracts, filings, and memoranda are primarily unstructured, making AI essential for efficient analysis and insight extraction.
Variable Auto Encoder (VAE):: A type of generative model that learns a low-dimensional, continuous latent representation of the input data, typically assuming a Gaussian prior distribution. VAEs consist of an encoder that maps the input to the latent space and a decoder that reconstructs the input from the latent representation.
Vector Database:: A database optimized for storing and searching high-dimensional vectors, which are numerical representations of data points. Vector databases enable efficient similarity search and retrieval of unstructured data, such as finding related documents or clauses.
Vector Embeddings:: Dense, continuous vector representations that capture the semantic meaning and relationships of data points, such as words, sentences, or documents. AI models use vector embeddings to understand the meaning and context of language, enabling tasks like document comparison and question answering.
Weights:: The learnable parameters of a machine learning model that determine the strength of connections between nodes in a neural network. During training, the model adjusts its weights to minimize the loss function and improve its predictions on tasks.
Zero-Shot Learning:: A learning paradigm where a model is trained to understand tasks or recognize objects during inference that it has never seen during training, relying on understanding the relationships between concepts.