Vector

In mathematics, a vector is a quantity that carries both magnitude and direction. It is usually written as an ordered list of numbers, for example

v = (3, 4)

where each number is called a component. The concept scales naturally to any number of dimensions: a 768‑dimensional vector might represent the embedding of a sentence, while a 1536‑dimensional vector could encode an image feature map. The counterpart of a vector is a scalar, a single number that has magnitude but no direction.

Vectors are the backbone of many scientific disciplines. In geometry they describe displacements; in physics they represent forces; in computer science they become data structures; and in artificial intelligence they encode semantic meaning.

Fundamental Operations

Addition and Scalar Multiplication

Adding two vectors is performed component‑wise:

(1, 2) + (3, 4) = (4, 6)

Multiplying a vector by a scalar scales every component:

2 × (1, 2) = (2, 4)

These operations preserve the dimensionality of the vectors involved.

Dot Product

The dot product (or inner product) measures how aligned two vectors are:

a · b = a₁×b₁ + a₂×b₂ + … + aₙ×bₙ

For instance, (1, 2) · (3, 4) = 11. The dot product is central to computing angles, projecting one vector onto another, and measuring similarity in high‑dimensional spaces.

Norm and Distance

The Euclidean norm (length) of a vector v is

‖v‖ = √(v₁² + v₂² + … + vₙ²)

The Euclidean distance between two vectors a and b is

‖a - b‖

Other distance metrics, such as cosine similarity, are also common in data‑driven applications.

Vectors in Programming

Dynamic Arrays in C++

The C++ Standard Library offers std::vector, a resizable array that stores elements contiguously:

#include <vector>
#include <iostream>

int main() {
    std::vector<int> nums = {1, 2, 3};
    nums.push_back(4);   // append at the end
    nums[0] = 10;        // random access, O(1)

    for (int x : nums) {
        std::cout << x << " ";
    }
    return 0;
}

Key properties:

• Random access in constant time.
• Amortized O(1) push/pop at the back.
• O(n) for insertions or deletions in the middle.
• Elements are stored contiguously, which improves cache locality compared to linked lists.

Arrays in Python and NumPy

Python’s built‑in list behaves similarly to std::vector, but for numerical workloads the numpy.ndarray is preferred:

import numpy as np

a = np.array([1, 2, 3])
b = np.array([4, 5, 6])

print(a + b)         # [5 7 9]
print(np.dot(a, b))  # 32

NumPy’s vectorized operations are implemented in C, offering substantial speedups over pure Python loops.

Vectors in Machine Learning

Embedding Vectors

Deep learning models transform raw data—text, images, audio—into dense, high‑dimensional vectors. These embedding vectors capture semantic relationships:

• Words that appear in similar contexts end up close together in the vector space.
• Images with similar visual patterns cluster in the same region.

Similarity Measures

Common metrics for comparing embeddings include:

Vector Databases and ANN

Large‑scale retrieval of the nearest vectors is a core operation in recommendation engines, semantic search, and retrieval‑augmented generation (RAG). Vector databases such as Milvus, FAISS, and Pinecone implement Approximate Nearest Neighbor (ANN) algorithms to return the top‑k most similar vectors from millions or billions of entries in milliseconds.

In a RAG pipeline, a user query is encoded into a vector, the database returns the closest knowledge‑base snippets, and a language model generates a response conditioned on those snippets.

Practical Applications

• Search Engines: Indexing documents as vectors enables semantic search beyond keyword matching.
• Recommendation Systems: User and item embeddings are compared to suggest relevant content.
• Computer Vision: Feature vectors extracted by convolutional networks power image retrieval.
• Natural Language Processing: Word, sentence, and document embeddings support tasks like classification, clustering, and translation.
• Robotics: Vectors represent positions, velocities, and forces in control algorithms.

Takeaway

Vectors unify concepts across disciplines:

• Mathematics: Geometric objects that describe direction and magnitude.
• Programming: Efficient, resizable containers that support fast arithmetic.
• Artificial Intelligence: Compact numerical encodings of meaning, enabling similarity search and reasoning.

Mastering vector notation, operations, and their software implementations lays the groundwork for deeper study in linear algebra, algorithm design, and modern machine learning. Experiment with the code snippets above, generate your own embeddings, and observe how distance metrics reveal hidden relationships in data.