Basic Tasks

Computer vision is not a single problem but a collection of interconnected tasks that teach machines to interpret visual information. Raw images are simply grids of numerical values; useful applications require systems to recognize objects, locate them, describe their shapes, follow them through time, or even create new visual content.

These tasks can be understood along two dimensions: the granularity of the output and the amount of context the model must consider. Classification produces a label for an entire image, detection adds spatial location, segmentation assigns a class to individual pixels, and tracking extends that understanding across frames. Other tasks focus on geometric structure, motion, or visual synthesis.

The differences among these tasks are often best understood through their output spaces. A classifier maps an image to a discrete category. A detector predicts a set of structured boxes. A segmentation model produces dense masks. A tracker estimates how identities persist across time. As the output becomes more detailed and structured, the required training data and modeling capacity generally increase.

Recognizing and Locating Visual Content

Image classification is usually the starting point for learning computer vision. Given an image, the model selects one or more labels, such as “cat,” “car,” or “tumor present.” Its central challenge is not merely identifying obvious objects, but separating categories that share many visual features. Fine-grained classification, for example, must distinguish dog breeds, bird species, or product variants whose differences may occupy only a few pixels.

Object detection adds location to recognition. Instead of assigning one label to the entire frame, a detector identifies multiple objects and estimates their positions, usually with bounding boxes. This makes detection well suited to scenes containing several objects at different scales. Modern systems combine feature extraction with region proposal, classification, and box-regression components, although contemporary architectures often frame these stages jointly.

The boundary between detection and segmentation is therefore one of precision rather than purpose. A box tells the system that a pedestrian occupies a particular region; a segmentation mask specifies exactly which pixels belong to that pedestrian. The latter is essential when precise outlines matter, such as separating tissue from healthy cells in a medical scan or isolating a person from the background in an editing application.

Extracting Structure and Understanding Motion

Not every vision task depends on recognizing complete object categories. Keypoint detection identifies meaningful points such as facial landmarks, joint centers, or corners of a geometric object. These points provide a compact structural representation and often serve as inputs to higher-level systems. Pose estimation, for example, may use detected joints to analyze exercise form, guide character animation, or support gesture-based interfaces.

Edge detection is an even more fundamental form of structure extraction. It identifies rapid changes in brightness or color that often correspond to object boundaries. Although modern neural networks can learn such features implicitly, edges remain important in classical pipelines, image registration, industrial inspection, and three-dimensional reconstruction.

Object tracking introduces time. Given observations across successive frames, the system estimates the position and identity of a target. Tracking is commonly built on top of detection, but the tasks are not identical: detection asks what is visible in one frame, while tracking asks whether several observations refer to the same entity over time. The difficult cases include occlusion, abrupt movement, similar-looking targets, and targets that leave and re-enter the scene.

Action recognition goes further by interpreting temporal patterns rather than merely maintaining identity. It combines appearance with cues such as direction, speed, and the order of movements. A person who is stationary may be waiting, while the same person walking, running, or falling may produce different motion patterns even though their appearance remains similar.

Generation and Restoration

Generative and restoration-based tasks move beyond direct interpretation of an input image. Image generation synthesizes new content from text prompts, examples, random noise, or other conditioning signals. Super-resolution increases spatial detail, while image restoration removes degradation caused by noise, blur, compression, or poor lighting.

Although these tasks are often introduced separately from recognition, the underlying boundary is less rigid than it may appear. A generative model must learn a detailed representation of objects, scenes, and their relationships to produce coherent images. Conversely, recognition systems can benefit from representations learned through large-scale generative pretraining. Understanding and synthesis therefore reinforce one another.

How the Tasks Work Together

Fundamental vision tasks rarely function as isolated tools. Classification provides many detection systems with their category-prediction heads and supplies feature-extraction backbones for more complex models. Detection can precede instance segmentation, where each detected object is associated with a pixel-level mask. It is also a common first stage in multi-object tracking, where observations must be linked across frames.

Low-level structural features remain useful beyond classification. Edges, corners, and keypoints support camera calibration, image alignment, visual odometry, and three-dimensional scene reconstruction. In a self-driving vehicle, for example, cameras may contribute detected vehicles and pedestrians, lane and road-segmentation masks, and temporal estimates of nearby objects. These outputs are fused into a coherent scene representation before a planning system decides how the vehicle should move.

Applications Across Industries

The appropriate task depends on the kind of answer an application needs:

• Classification supports content moderation, document sorting, medical triage, and automated quality inspection.
• Detection is widely used in autonomous driving, video surveillance, retail analytics, and search-based image indexing.
• Segmentation assists medical-image outlining, remote-sensing analysis, background replacement, and precision agriculture.
• Keypoint detection enables fitness coaching, human–computer interaction, facial alignment, and animation production.
• Tracking supports traffic analysis, sports analytics, drone cinematography, and security monitoring.
• Action recognition applies to industrial safety, surveillance, sports analysis, and human–machine interaction.
• Generation and restoration power creative tooling, old-photo enhancement, low-light photography, visual effects, and synthetic-content production.

These categories also overlap in practice. A retail system might classify a product, detect its position on a shelf, segment the package outline, and track customer attention over time.

A Practical Learning Path

A productive learning sequence begins with classification, because it introduces the basic workflow of data preparation, feature learning, optimization, and evaluation. Object detection comes next, teaching students how recognition and localization can be optimized together. Segmentation then introduces dense prediction and exposes the practical demands of pixel-level annotation. Tracking expands the framework from spatial reasoning to temporal consistency.

From there, learners can specialize according to their interests. Keypoint and geometry tasks offer entry points into robotics and three-dimensional vision. Action recognition leads toward video understanding. Generative models open routes into content creation, representation learning, and creative AI. Regardless of specialization, understanding how the major tasks relate provides a durable foundation rather than a narrow collection of model recipes.

Ultimately, computer vision forms a continuum. Classification answers what is present, detection answers where it is, segmentation answers which pixels belong to it, and tracking asks how it changes over time. Generation extends the discipline by asking what visual content can be produced. Mastering these distinctions equips learners to select suitable methods, combine them into larger systems, and reason clearly about the many ways machines can interpret the visual world.