EPIC-KITCHENS

EPIC-KITCHENS is a dataset of first-person (egocentric) video recorded with head-mounted cameras by people in their own kitchens, without a script, and labelled from the participants’ own spoken narrations. It was built by Dima Damen’s group at the University of Bristol with the University of Catania and, for the first version, the University of Toronto. The first version, released in April 2018 and later named EPIC-KITCHENS-55 after its 55 hours, was described at ECCV 2018 and in an extended TPAMI paper (2021). The extension EPIC-KITCHENS-100, released on 1 July 2020 and described in IJCV (2022), brings the footage to 100 hours. Its central label is the action, a pair of a verb class and a noun class such as (open, fridge), which makes it a benchmark for fine-grained action recognition of hand–object interactions, and also for anticipating, detecting and retrieving actions.

Purpose

The ECCV paper names two gaps in earlier data. Egocentric datasets were small, often recorded in a single environment, and scripted: people were told which actions to perform, so they performed them in order. Larger third-person datasets of daily activity, such as Charades, were also scripted, which the authors argue makes actions look less natural and removes the multi-tasking of real life. EPIC-KITCHENS asked participants to record every visit to their kitchen for three consecutive days (“at least three” in the TPAMI version). The footage shows searching for items, changing one’s mind and washing a few dishes while cooking. The authors also chose narration by the participants themselves, as the people best placed to say what they were doing, and to reflect intention.

The IJCV paper adds a second aim for the 100-hour version: one dataset with many tasks on the same input. The narrations, timestamps, segments and classes support six challenges, and recording two years later supports a “test of time” for models trained on the 2018 footage.

Dataset Size

	EPIC-KITCHENS-55	EPIC-KITCHENS-100
Recorded	May–November 2017	2018 footage plus an extension recorded about two years later
Participants, kitchens	32, 32	37, 45
Hours	55	100.0 (54.6 from the 2018 videos, 45.4 new)
Videos	432 sequences	700 (432 + 268)
Frames	11.5M	20M
Action segments	39,564 (text); 39,596 narrations (Table 1)	89,977
Verb, noun classes	125, 331	97, 300
Action classes	149 with more than 50 samples (Table 1)	4,053
Object annotations	454,255 boxes (ECCV)	66M masks, 31M hand and 38M object boxes, automatic

The EPIC-KITCHENS-55 figures are from the ECCV paper; the EPIC-KITCHENS-100 figures from the IJCV paper’s Table 1 and abstract. The sources disagree in small ways. The ECCV abstract and Table 1 count 39.6K segments and 39,596 narrations, while the text counts 39,564 segments, the 99.9% of narrations that annotators found visible; the official EPIC-KITCHENS-55 page lists 39,594 (as of October 2026). The TPAMI paper (arXiv version) gives 454,158 boxes in Table 1 and 454.2K in the abstract, against 454,255 in the ECCV paper and on the official page. EPIC-KITCHENS-100 re-parsed the 2018 narrations with its new pipeline, so the 2018 videos contribute 39,432 segments to it rather than the original count.

The 2018 footage was filmed on a head-mounted GoPro (a Hero 5, per the data deposit’s readme) at 1920 × 1080 and 59.94 fps, with a few videos at other resolutions and frame rates. The extension used a GoPro Hero7 Black at 50 fps with built-in stabilization. The IJCV appendix says 50 fps avoids the flicker of overhead lights seen in the 2018 videos. Participants recorded on average 1.7 hours each in 2017, at most 4.6 hours. The 2017 recordings came from Bristol (15 participants), Toronto (8), Catania (8) and Seattle (1). Of the original 32, 16 recorded again for the extension, half of them in a new home, and 5 new participants joined.

Classes

Classes come from the narrations, not from a fixed list. For EPIC-KITCHENS-55 the authors parsed each transcribed narration with spaCy, took the first verb and the nouns, and grouped them into classes with minimal semantic overlap. They clustered verbs by hand and nouns semi-automatically, after automatic clustering with WordNet and Word2Vec produced too many meaningless groups. Class 0, take, for example, collects take, grab, pick, get and fetch. When a narration names several objects, the action’s noun is the first one; when the noun is missing or a pronoun, the noun of the previous narration is used.

EPIC-KITCHENS-100 re-parsed all narrations, the 2018 ones included, and re-clustered to reduce ambiguity: brush and sweep became one verb class, and new noun classes such as lentils appeared. The result is 97 verb classes in 13 categories and 300 noun classes in 21 categories. Both versions are long-tailed (ECCV, Fig. 7; IJCV, Fig. 4). Only combinations that occur are action classes, so EPIC-KITCHENS-100’s 4,053 actions are a small part of the 29,100 possible pairs.

Annotation Types

Narrations. In EPIC-KITCHENS-55, each participant watched their videos after recording and narrated them like a live commentary into a hand-held recorder, in English or their native language. Seventeen narrated in English, seven in Italian, six in Spanish, one in Greek and one in Chinese. Speech-to-text services failed on these short phrases, so Amazon Mechanical Turk (AMT) workers transcribed and translated them. Each task was done three times and a transcription kept when it matched another exactly, and YouTube’s caption alignment supplied timestamps. A narration was recorded on average every 4.9 seconds, with 2.8 words per phrase.

For EPIC-KITCHENS-100 the authors replaced the running commentary with a “pause-and-talk” narrator, an open-source tool in which the participant holds a key to pause the video while speaking. Narrating the previous action while the next ones played had caused missed and misspoken actions. Transcribers now saw three frames around each timestamp, which resolved homonyms such as “flower” and “flour”; three workers transcribed each narration, with automatic rejection below a Word2Vec similarity of 0.9. A hired translator then translated the unique transcriptions. On two equal-length stretches narrated by the same participant, the new narrator raised the number of actions from 20 to 56, short ones such as turning on a tap included (IJCV, Fig. 3).

Action segments. AMT workers turned each narration timestamp into start and end times. In EPIC-KITCHENS-55, four workers labelled each segment. The final segment is that of the worker who agrees most with the others, measured by temporal IoU, merged with the closest other annotation when the two overlap with IoU above 0.5 (ECCV, Eq. 1). EPIC-KITCHENS-100 used five workers and started them from the pause-and-talk timestamp. Segments may overlap; segments averaged 3.7 s (standard deviation 5.6 s) in the first version and 3.1 ± 5.4 s in the second.

Objects. EPIC-KITCHENS-55 has boxes around the narrated “active” objects in frames from 2 s before to 2 s after each segment, sampled at 2 fps, from three AMT workers per task. EPIC-KITCHENS-100 adds automatic annotations instead: Mask R-CNN masks from a COCO-trained model and hand–object detections. The authors did not evaluate the masks and call them error-prone.

Quality. On 300 random EPIC-KITCHENS-55 samples, the authors found errors in 5.7% of segment boundaries, 6.3% of boxes, 3.3% of verb classes and 6.0% of noun classes.

Splits

EPIC-KITCHENS-55 held out ground truth for 27% of the data in two test sets (ECCV, Table 4):

Split	Participants	Sequences	Action segments
Train	28	272	28,561
S1, seen kitchens	28	106	8,064
S2, unseen kitchens	4	54	2,939

In S1, every kitchen appears in both training and testing, with roughly 80% of sequences in training and 20% in testing, and whole sequences on one side. S2 holds out all sequences of four participants, 7% of the frames. The TPAMI version counts zero-shot classes in each test set: in S2, 1 verb, 13 nouns and 220 actions never occur in training, mostly new combinations of verbs and nouns seen separately.

EPIC-KITCHENS-100 splits whole videos into Train, Val and Test in roughly 75/10/15 proportions: 67,217, 9,668 and 13,092 segments. The Test split contains only newly recorded videos; re-parsed videos from the 2018 test sets became the validation set. The seen/unseen test split was dropped. Two subsets are reported instead:

• Unseen participants. Two participants appear only in Val and three only in Test, with 1,065 and 4,110 segments.
• Tail classes. For verbs and for nouns, the smallest classes whose instances add up to 20% of the training instances: 86 of 97 verbs and 228 of 300 nouns. A tail action has a tail verb or a tail noun; there are 3,729 such action classes.

Test labels are withheld for the challenge servers. The annotation repository publishes the lists of unseen participants and tail classes. Two challenges use their own splits. Unsupervised domain adaptation trains on labelled 2018 videos of 16 participants and adapts to unlabelled 2020 videos of the same people. Retrieval uses Val as its test set, so that the captions of the main Test set stay hidden.

Tasks

EPIC-KITCHENS-55 defined three challenges: object detection, action recognition and action anticipation. EPIC-KITCHENS-100 defined six:

• Action recognition: classify a trimmed segment into a verb, a noun and their action.
• Weakly supervised action recognition: the same test, but training gives only the narration’s single timestamp per action, not its start and end.
• Action detection: find the start, end, verb and noun of every action in an untrimmed video (temporal action localization).
• Action anticipation: predict the next action from video that ends $\tau_a = 1$ s before it starts.
• Unsupervised domain adaptation: recognize actions in unlabelled 2020 footage after training on labelled 2018 footage.
• Multi-instance retrieval: rank captions for a video segment, or segments for a caption (image–text retrieval for video).

Challenge results have been announced each year since 2019 at workshops held with CVPR (official challenge pages). As of October 2026, the 2026 page lists recognition, detection, domain adaptation and retrieval, alongside segmentation and audio challenges built on later annotations; anticipation is last listed among the challenges on the 2024 page, although the 2025 page still describes it.

Evaluation

Action recognition: verb, noun and action accuracy

A submission gives, for each segment, a score for every verb class and every noun class. The official C1 code at commit 277586f (evaluate.py, with utils/scoring.py and utils/metrics.py), which was run on the example below, computes the metrics as follows:

Action scores. Each head’s 100 highest scores are turned into probabilities with a softmax, and every (verb, noun) pair among them gets the product of its two probabilities:

$$s(v, n) = \frac{e^{x_v}}{\sum_{v' \in T_V} e^{x_{v'}}} \cdot \frac{e^{y_n}}{\sum_{n' \in T_N} e^{y_{n'}}},$$

where $x$ and $y$ are the submitted verb and noun scores and $T_V$, $T_N$ the top-100 sets, which include all 97 verbs. The 100 best pairs are kept, encoded as $1000\,v + n$. The IJCV appendix describes the baselines’ action predictions the same way, as combinations of softmaxed verb and noun scores.

Accuracy. Top-1 and top-5 accuracy for verbs, nouns and actions: a segment counts as correct at $k$ if its true class is among the $k$ best-ranked. The mean is over segments, so frequent classes dominate.

Subsets. Unseen-participant accuracy uses the segments whose participant ID is in the unseen list; tail accuracy uses, for verbs, segments with a tail verb, for nouns, segments with a tail noun, and for actions, segments with either. Both subsets report top-1 only.

Three consequences follow from the code. Since the product of two softmaxes is largest for the best verb paired with the best noun, the top-1 action is always that pair, whatever the scores’ scale; action top-1 is therefore exactly the fraction of segments in which both heads are right. Second, the softmax is applied to the submitted scores, so they are expected to be logits; scores that are already probabilities are softmaxed again, which keeps top-1 but can reorder the top-5 actions (our reading). Third, scipy.special.softmax is called without an axis, so the denominators in the formula above run over the top scores of all segments at once, not of one segment. Rankings within a segment are unchanged, but with float32 inputs, a segment whose verb and noun scores lie, added together, roughly 100 or more below the submission’s highest ones (for example 50 below in each head) gets action scores that underflow to zero and an arbitrary action ranking (our reading of the code, confirmed on synthetic scores). The README says the script computes the metrics reported in the paper on the validation set; we did not find the scoring code of the test server.

EPIC-KITCHENS-55 used the same accuracies, computed per test set (S1 and S2), and added many-shot precision and recall. In the epic-kitchens-55-lib code (commit 7f2499a, read line by line; its np.float annotations fail to import from NumPy 1.24 on), each class’s precision and recall are taken from top-1 predictions with scikit-learn, then averaged over the many-shot classes present in the test set. Many-shot verbs and nouns have more than 100 training instances, and many-shot actions are pairs of them seen at least once in training. The ECCV paper and the many-shot-classes repository count 26 verbs, 71 nouns and 819 actions; the TPAMI arXiv version says 27 verbs, 67 nouns and 758 actions. The ECCV text words the 100-sample rule as if it applied to actions too, while the code and the TPAMI text use combinations of many-shot verbs and nouns. The ECCV paper notes that the per-class metrics of smaller classes are close to zero for its baselines. The library can also multiply action scores by an action prior, such as training counts of each pair.

Example: why action accuracy is low

The code builds synthetic segments with EPIC-like structure: 30 verbs with Zipf frequencies, 80 nouns, each verb used with eight nouns, 12 kitchens in training and 3 more only in testing. Two simulated classifier heads score verbs and nouns. The true class gets a margin that shrinks for rare classes, for unseen kitchens and for hard segments, and in the default setting both heads share each segment’s difficulty, as blur or occlusion would. The metrics follow the C1 code; class-mean top-5 recall, used for anticipation below, is computed too.

import numpy as np

V, N = 30, 80                              # verb and noun classes
SEEN, UNSEEN = range(12), range(12, 15)    # kitchens in training; kitchens only in testing
PAIRS = np.random.default_rng(123).choice(N, size=(V, 8))   # nouns each verb is used with

def make_segments(rng, n_seg, kitchens):
    """Action segments: a long-tailed (Zipf) verb, then one of that verb's eight nouns,
    weighted by a long-tailed noun frequency."""
    p_verb = 1.0 / np.arange(1, V + 1) ** 1.2
    p_noun = 1.0 / np.arange(1, N + 1)
    v = rng.choice(V, n_seg, p=p_verb / p_verb.sum())
    w = p_noun[PAIRS[v]].cumsum(1)
    n = PAIRS[v, (w / w[:, -1:] > rng.random((n_seg, 1))).argmax(1)]
    return v, n, rng.choice(kitchens, n_seg)

def tail_classes(train_labels, n_classes, share=0.2):
    """Smallest classes whose instances add up to 20% of the training instances."""
    counts = np.bincount(train_labels, minlength=n_classes)
    order = np.argsort(counts, kind="stable")
    return order[np.cumsum(counts[order]) <= share * counts.sum()]

def head_logits(rng, labels, train_counts, unseen, difficulty, drop):
    """One classifier head: noise plus a frequency prior, and a margin on the true class
    that shrinks for rare classes, unseen kitchens and hard segments."""
    margin = 0.32 * np.log1p(train_counts[labels]) * difficulty * np.where(unseen, drop, 1.0)
    logits = rng.normal(0, 1, (len(labels), len(train_counts))) + 0.3 * np.log1p(train_counts)
    logits[np.arange(len(labels)), labels] += margin
    return logits

def ranks(scores):                         # classes in decreasing order of score
    return np.argsort(-scores, axis=1, kind="stable")

def action_ranks(verb_scores, noun_scores, top_n=100):
    """As the official C1 code: softmax over each head's top-100 scores, multiply,
    keep the 100 best (verb, noun) pairs, encoded as verb * 1000 + noun."""
    tv, tn = ranks(verb_scores)[:, :top_n], ranks(noun_scores)[:, :top_n]
    pv, pn = (np.exp(s - s.max(1, keepdims=True)) for s in
              (np.take_along_axis(verb_scores, tv, 1), np.take_along_axis(noun_scores, tn, 1)))
    joint = (pv[:, :, None] / pv.sum(1)[:, None, None] * pn[:, None, :] / pn.sum(1)[:, None, None])
    i, j = np.unravel_index(ranks(joint.reshape(len(tv), -1))[:, :top_n], joint.shape[1:])
    rows = np.arange(len(tv))[:, None]
    return tv[rows, i] * 1000 + tn[rows, j]

def topk(r, labels, k):
    return np.mean((r[:, :k] == labels[:, None]).any(1))

def mean_top5_recall(r, labels):           # anticipation: top-5 recall averaged over classes
    return np.mean([topk(r[labels == c], labels[labels == c], 5) for c in np.unique(labels)])

def simulate(seed, shared_difficulty=True, n_test=6000):
    rng = np.random.default_rng(seed)
    tr_v, tr_n, _ = make_segments(rng, 20000, list(SEEN))
    te_v, te_n, te_k = make_segments(rng, n_test, list(SEEN) + 2 * list(UNSEEN))
    unseen = np.isin(te_k, list(UNSEEN))
    hard_v = rng.uniform(0.2, 1.8, n_test)  # per-segment difficulty: blur, occlusion, ...
    hard_n = hard_v if shared_difficulty else rng.uniform(0.2, 1.8, n_test)
    vs = head_logits(rng, te_v, np.bincount(tr_v, minlength=V), unseen, hard_v, drop=0.8)
    ns = head_logits(rng, te_n, np.bincount(tr_n, minlength=N), unseen, hard_n, drop=0.6)
    return tr_v, tr_n, te_v, te_n, unseen, vs, ns

tr_v, tr_n, te_v, te_n, unseen, vs, ns = simulate(seed=0)
labels = {"verb": te_v, "noun": te_n, "action": te_v * 1000 + te_n}
R = {"verb": ranks(vs), "noun": ranks(ns), "action": action_ranks(vs, ns)}
tail = {"verb": np.isin(te_v, tail_classes(tr_v, V)), "noun": np.isin(te_n, tail_classes(tr_n, N))}
tail["action"] = tail["verb"] | tail["noun"]   # a tail action has a tail verb or a tail noun

print(f"{len(te_v)} test segments, {unseen.sum()} from unseen kitchens, "
      f"{len(np.unique(labels['action']))} distinct actions; tail classes: "
      f"{len(tail_classes(tr_v, V))} verbs, {len(tail_classes(tr_n, N))} nouns")
print("        top-1  top-5  unseen top-1  tail top-1  class-mean top-5 recall")
for t in R:
    r, y, u, tl = R[t], labels[t], unseen, tail[t]
    print(f"{t:<7} {topk(r, y, 1):.3f}  {topk(r, y, 5):.3f}  {topk(r[u], y[u], 1):.3f}"
          f"         {topk(r[tl], y[tl], 1):.3f}       {mean_top5_recall(r, y):.3f}")
both = (R["verb"][:, 0] == te_v) & (R["noun"][:, 0] == te_n)
print(f"verb top-1 x noun top-1 = {topk(R['verb'], te_v, 1) * topk(R['noun'], te_n, 1):.3f}; "
      f"verb and noun both right = {both.mean():.3f}")

for shared in (True, False):               # action top-1 = both heads right; compare to product
    gaps = []
    for seed in range(10):
        _, _, v, n, _, a, b = simulate(seed, shared)
        hit_v, hit_n = a.argmax(1) == v, b.argmax(1) == n
        gaps.append((hit_v & hit_n).mean() - hit_v.mean() * hit_n.mean())
    print(f"{'shared' if shared else 'independent'} difficulty, seeds 0-9: "
          f"action top-1 - verb x noun = {min(gaps):+.3f} to {max(gaps):+.3f}")

Output:

6000 test segments, 1988 from unseen kitchens, 220 distinct actions; tail classes: 20 verbs, 59 nouns
        top-1  top-5  unseen top-1  tail top-1  class-mean top-5 recall
verb    0.611  0.834  0.548         0.315       0.674
noun    0.438  0.663  0.288         0.154       0.378
action  0.335  0.510  0.203         0.134       0.250
verb top-1 x noun top-1 = 0.268; verb and noun both right = 0.335

shared difficulty, seeds 0-9: action top-1 - verb x noun = +0.067 to +0.078
independent difficulty, seeds 0-9: action top-1 - verb x noun = +0.016 to +0.025

Action top-1 equals the fraction of segments with both heads right, as the code implies, so it cannot exceed the lower of verb and noun accuracy. How far below that it falls depends on how the two heads’ errors relate. If they were independent, action accuracy would be the product of verb and noun accuracy, 0.268 here. It is higher, 0.335, because the heads succeed and fail on the same segments: over seeds 0–9 the gap to the product is +0.067 to +0.078 when difficulty is shared. With each head drawing its own difficulty the gap shrinks to +0.016 to +0.025. It does not vanish, because unseen kitchens and rare classes still make both heads worse on the same segments. The printed EPIC-KITCHENS-100 baselines show the same pattern: TSN on Val has 60.18% verb and 46.03% noun top-1, whose product is 27.7%, while its printed action top-1 is 33.19% (IJCV, Table 3; the product is our arithmetic). Top-5 action accuracy is low for a second reason: five guesses among 2,400 (verb, noun) pairs, against five among 30 verbs.

The subsets behave as their definitions predict. The unseen-kitchen drop was put into the simulation, larger for nouns than verbs, so it only shows that the subset isolates it. The tail rule selects 20 of 30 verbs and 59 of 80 nouns, most classes, as it does on EPIC-KITCHENS-100 (86 of 97 verbs, 228 of 300 nouns). Accuracy on tail segments is roughly a third to a half of the overall value. Class-mean top-5 recall weights every class equally, so for nouns and actions it falls well below top-5 accuracy, which the head classes dominate.

The official functions, imported from the pinned commits, gave identical numbers on the same arrays. compute_metrics and compute_action_scores from C1 matched every accuracy above, with the same top-5 action lists, and mean_topk_recall from C3 matched the class-mean recalls.

Action anticipation

EPIC-KITCHENS-55 scored anticipation with the recognition metrics, on segments observed for $\tau_o = 1$ s ending $\tau_a = 1$ s before each action. EPIC-KITCHENS-100 lets methods observe any length of video ending 1 s before the action and reports class-mean top-5 recall: for each class, the fraction of its segments whose true class is among the five best predictions, averaged over classes. Furnari, Battiato and Farinella (2018) defined this top-$k$ recall per class for anticipation. The IJCV paper argues that top-5 allows for the uncertainty of the future and that averaging over classes balances the long tail. In the C3 evaluation code (commit 73b8a00, run on the example above), the average runs over the classes present in the evaluated set or subset. Action scores are formed as in C1, but with a softmax over each segment’s own scores, unless the submission provides its own 100 action scores. The same unseen-participant and tail subsets are reported, also as class-mean top-5 recall.

Action detection

The C2 evaluation code (commit 4835721, read line by line; it uses np.float and does not run from NumPy 1.24 on) is adapted from the ActivityNet detection evaluation (see ActivityNet). For verbs, nouns and actions separately, it matches predictions to ground-truth segments of the same class and video by temporal IoU, in order of decreasing score, at thresholds 0.1, 0.2, 0.3, 0.4 and 0.5 (a match needs tIoU at least the threshold), each ground-truth segment matched at most once. It computes interpolated average precision per class, averages over the classes present in the ground truth, and averages the five mAPs. Predictions of classes absent from the ground truth are dropped before scoring.

Object detection, retrieval and domain adaptation

EPIC-KITCHENS-55’s object-detection challenge evaluates each noun class only on images where that class was annotated, since only active objects carry boxes. It uses Pascal VOC mAP at IoU 0.05, 0.5 and 0.75, split into many-shot classes (at least 100 training boxes) and few-shot classes (10 to 99) (ECCV, Sec. 4.1). Multi-instance retrieval uses mAP and normalized discounted cumulative gain, whose relevance between two captions is the mean IoU of their verb and noun class sets (IJCV, Sec. 4.6). It thus treats “put glass” and “place cup” as relevant to each other, unlike retrieval benchmarks that count only a video’s own captions as relevant. Domain adaptation uses the recognition accuracies on the target test videos. Every EPIC-KITCHENS-100 submission also declares supervision levels for pretraining, training labels and training data, on a 0–5 scale, so that results using extra data are marked.

License

As of October 2026, the official page states that all datasets and benchmarks on it are copyright of the authors and published under the Creative Commons Attribution-NonCommercial 4.0 International License, and offers commercial licences by email. The annotation repositories and the readmes of both video deposits state the same licence. The University of Bristol’s data repository, however, lists the licence of both video deposits as the Non-Commercial Government Licence for public sector information (version 2), and the IJCV paper’s data release statement names that licence for the videos, frames and optical flow. Both licences exclude commercial use; for anything beyond non-commercial research, the terms should be confirmed with the authors. The footage was recorded by the participants for the project, so no third-party image licences are involved (our reading). The C1 evaluation repository is itself under CC BY-NC 4.0.

Limitations and Bias

People and privacy. The recordings show people’s homes and hands, so the dataset’s own statements on consent and privacy matter. In the 2017 collection, participants recorded voluntarily and were not paid. They were asked to be alone in the kitchen and to remove items that would reveal their identity, such as portraits or mirrors (ECCV, Sec. 3.1; TPAMI, same text). The TPAMI paper’s data access statement says the data are released with the relevant consent forms; both video deposits have a folder named ConsentForms, and the EPIC-KITCHENS-100 deposit also one named AcquisitionGuidelines (data.bris listings, as of October 2026; we did not open them). The IJCV paper thanks “all voluntary participants” who collected and narrated the dataset. The extension’s readme says GPS was not recorded, for anonymity, and is suppressed in the released metadata. We found no statement on blurring faces or other image redaction in the three papers, the official pages or the readmes as of October 2026. With a head-mounted camera and one person in the room, the wearer’s face is rarely in view; that is our reading, not a claim of the sources. The recording times released with the extension give the local date, hour and minute of each video.

Who and where. The 2017 participants came from 10 nationalities in four cities in Europe and North America. All footage is of kitchens, which the IJCV paper presents as a domain-specific focus. The official page’s disclaimer warns that the dataset “may have unintended biases” of a societal, gender or racial nature (as of October 2026).

Labels from narrations. Participants narrated selectively. They named “open” more often than “close”, by which time their attention had moved to the next goal, so the authors evaluate only narrated actions. Narrations lag the action, and free vocabulary forces a manual grouping into classes, whose boundaries are partly a matter of choice. The pause-and-talk narrator reduced missed actions but cannot remove them.

Long tail and shortcuts. The tail rule puts most classes in the tail, and accuracy over segments is dominated by a few frequent verbs such as take and put. The TPAMI paper’s time-of-day baseline predicts the most common class for each hour of recording and beats the largest-class baseline. Such priors can carry a model some way without visual understanding.

Domain shift. Models trained on the 2018 videos lost accuracy on the 2020 videos (IJCV, Fig. 6). The authors trace the gap to the new camera, homes that changed, new participants, and wear and tear and objects that moved over two years. Adding new training data helped, but the gains saturated at about half of it, especially for unseen participants.

Common Uses

• Fine-grained action recognition benchmark, in which an action is right only if both the motion (the verb) and the object (the noun) are. The baselines include TSN, TRN, TSM and SlowFast, and video transformers such as ViViT were evaluated on it.
• Anticipation, detection and weak supervision from the same annotations, with dense, overlapping actions: 128.5 instances and 53.2 classes per video on average (IJCV, Table 4).
• Audio-visual learning, since the videos carry sound. In the TPAMI baselines, a TSN on audio alone beat the RGB network on top-1 verb accuracy in unseen kitchens (Table 8).
• Domain adaptation across time, and retrieval with graded relevance.
• Source for further annotations by the same team, listed on the official landing page as of October 2026: VISOR (hand and active-object masks), EPIC-SOUNDS (audio events), EPIC-Fields (camera poses) and HD-EPIC (a separately recorded, highly detailed set).

Historical Milestones

• 2018. EPIC-KITCHENS released on 9 April (official page; the video deposit is dated 5 April 2018). The ECCV baseline, a two-stream TSN with separate verb and noun outputs, reached 48.23% verb, 36.71% noun and 20.54% action top-1 on S1, and 39.40%, 22.70% and 10.89% on S2 (ECCV, Table 6).
• 2019. First challenges, with results announced at CVPR 2019 workshops.
• 2020. The TPAMI arXiv version reports re-trained baselines; TSM reached 29.90% action top-1 on S1 and 17.38% on S2 (Table 9), and a TSN with RGB, flow and audio 26.06% on S1 (Table 8). EPIC-KITCHENS-100 was released on 1 July (official page; the video deposit is dated 15 May 2020). Its recognition baselines on Val reach 38.54% action top-1 for SlowFast and 38.27% for TSM; on Test, TSM reached 37.39% and SlowFast 36.81% (IJCV, Table 3).
• 2021. Unsupervised domain adaptation was run as a challenge; the IJCV paper reports that the best 2021 submissions used several modalities.

Related Datasets

Earlier egocentric datasets were smaller and mostly scripted or recorded in a single environment: ADL (2012), the closest in spirit with recordings in people’s homes, GTEA Gaze+ and EGTEA Gaze+, BEOID and Charades-Ego (ECCV, Table 1). Among third-person cooking datasets, MPII Cooking 2 and 50 Salads were recorded in a single kitchen, and Breakfast follows a list of breakfast recipes. Ego4D (Grauman et al., 2022) later collected 3,000 hours of first-person video of daily life well beyond the kitchen. Something-Something also targets hand–object interactions, but from scripted clips of single actions. UCF101, Kinetics and ActivityNet are third-person web video, and AVA is cut from films.

Resources

• EPIC-KITCHENS official site: the current page, downloads, challenge guidelines and links to the leaderboards.
• EPIC-KITCHENS-100 annotations: splits, verb and noun classes, unseen-participant and tail-class lists.
• C1-Action-Recognition, C2-Action-Detection and C3-Action-Anticipation: official evaluation code.
• EPIC-KITCHENS-100 narrator: the pause-and-talk annotation tool.
• IJCV paper (open access) and the ECCV 2018 paper.