Vision Systems & Engineering
Ethics of Computer Vision
The ethical questions that systems which see people raise (capture without consent, identification at scale, inference of what people do not show, errors that fall on people, dual use), the NIST AI Risk Management Framework and the EU AI Act in outline with their dates as of November 2026, research practice, and the limits of technical fixes.
beginner
Computer vision is the part of AI that most often looks at people: their faces, bodies, movements, homes and streets. Whether a system works is a question of accuracy; whether it should be built, on whose images, for what purpose and with whose consent are questions of ethics. This article maps those questions as they arise in vision, linking the articles that hold the measurements and the cases, and outlines the NIST AI Risk Management Framework and the EU AI Act as of November 2026. It draws no legal conclusions; every legal statement is attributed to a named document and date.
Definition
The NIST AI Risk Management Framework (AI RMF 1.0, January 2023) states that it is the joint responsibility of all AI actors to determine whether AI technology is an appropriate or necessary tool for a given context or purpose, and how to use it responsibly (NIST AI 100-1, §3). In our wording, the ethics of computer vision asks four things of a system that captures or interprets images of people: whether it should exist for its purpose, whose images it is built and run on and with what consent, what it infers about the people it sees, and who bears its errors and misuse. The framework itself is voluntary, “rights-preserving, non-sector-specific, and use-case agnostic” (Executive Summary); in our reading, law, where it applies, is a separate and narrower question.
Intuition
Consider one face-matching pipeline: detect a face, compute an embedding, compare it with stored ones (face recognition describes the steps). Used to unlock a phone, or at an airport e-gate that compares a traveller’s face with the passport image, it performs verification. Pointed at a crowd and compared with a watch list, it performs identification of people who did not step forward. The European Commission’s guidelines on prohibited AI practices (C(2025) 5052) treat the two differently: biometric verification falls outside the prohibition on real-time remote biometric identification, and confirming identity to access a service, unlock a device or enter premises is excluded from “remote” (paras 303–305), while remote identification is defined as identifying people without their active involvement, typically at a distance (para 298). In our reading, the code barely changes between the two uses; what changes is purpose, place and who agreed, and those are what ethics and law attend to.
Questions Specific to Vision
- People captured without being asked. Cameras record bystanders as well as subjects, as the privacy article describes. Web-scale datasets are built from photographs whose subjects were not asked: LAION reports that LAION-5B’s datasheet says people shown in its images may not have consented to their upload, and dataset bias describes the datasheet questions on notification, consent and its revocation. The Commission’s guidelines state that a person’s publishing facial images of themselves on social media does not mean they consented to those images being included in a facial recognition database (para 232).
- Identification and tracking at scale. Person re-identification links people across cameras, and video surveillance applies it across a site; that article summarizes the European data protection authorities’ statements. The guidelines, summarizing Recital 32 of the AI Act, say real-time remote biometric identification in publicly accessible spaces for law enforcement purposes may affect the private life of a large part of the population, “evoke a feeling of constant surveillance” and indirectly dissuade the exercise of freedom of assembly (para 293); systems used for following people can count as biometric identification (para 302).
- Inferring what people do not show. A system can assign people to categories from their appearance. The guidelines give attribute estimation that counts age, gender or ethnicity from the face or body as an example of individual biometric categorisation (para 282). On emotion, they quote Recital 44’s “serious concerns about the scientific basis” of systems that identify or infer emotions, since expression varies across cultures, situations and within one person (para 241), and they separate observation from inference: noting that a person is smiling is not emotion recognition, concluding that the person is happy is, and inferring a driver’s fatigue is not, because fatigue is a physical state (para 249).
- Errors that fall on people. When a system’s errors differ between groups, the harm is uneven. Bias and fairness defines the criteria and shows why they cannot all hold at once; face recognition and biometrics report NIST’s demographic differentials. The guidelines note that technical inaccuracies of remote biometric identification can produce biased results, particularly with regard to age, ethnicity, race, sex or disabilities (para 293).
- Dual use. The same methods serve benign and harmful ends. Deepfakes calls deepfakes one of computer vision’s clearest cases of dual use and documents both uses and harms; remote sensing notes that xView was developed with defence organizations; image editing reports method authors’ own impact statements on misuse. The AI RMF’s MAP function includes anticipating risks of the use of AI systems beyond their intended use (§5.2).
- The labour behind the labels. Training data are labelled by people whose pay and conditions vary widely; image annotation reports what studies and dataset cards disclose.
Frameworks in Outline
NIST AI RMF 1.0
The framework names seven characteristics of trustworthy AI: valid and reliable, safe, secure and resilient, accountable and transparent, explainable and interpretable, privacy-enhanced, and fair with harmful bias managed. Valid and reliable is the base on which the others rest, and accountable and transparent relates to all of them (§3, Fig. 4). The characteristics trade off against each other, and trustworthiness “is only as strong as its weakest characteristics” (§3). The privacy, fairness and testing articles summarize the sections on privacy (§3.6), fairness (§3.7) and validity with test, evaluation, verification and validation (§3.1).
Its core has four functions (§5). GOVERN cultivates a culture of risk management and runs through the other three. MAP establishes the context and should leave enough knowledge of impacts to inform “an initial go/no-go decision” about whether to design, develop or deploy a system. MEASURE analyses and monitors risk; systems should be tested before deployment and regularly in operation, and independent review can mitigate internal biases and conflicts of interest. MANAGE allocates resources to the risks found, including mechanisms to “supersede, disengage, or deactivate” systems whose outcomes are inconsistent with their intended use (MANAGE 2.4). Where risks are unacceptable, development and deployment should cease in a safe manner until the risks can be managed (§1.2.3).
The EU AI Act as the Commission describes it
The European Commission’s AI Act page, as read on 14 November 2026 (last updated 19 October 2026), describes the Act, Regulation (EU) 2024/1689, as defining four levels of risk: unacceptable, high, transparency and minimal or no risk. Its text was not read for this article.
- Prohibited practices. The page says the Act “prohibits nine practices”. Among them are untargeted scraping of the internet or CCTV material to create or expand facial recognition databases, emotion recognition in workplaces and education institutions, biometric categorisation to deduce certain protected characteristics, real-time remote biometric identification for law enforcement purposes in publicly accessible spaces, and AI systems that generate non-consensual sexually explicit and intimate content or child sexual abuse material. “Prohibitions 1-8 became effective in February 2025”; prohibition 9 “comes into effect in December 2026” and was introduced as part of the AI Omnibus. The Commission’s guidelines on prohibited practices, dated 29 July 2025 on their title page and shown as “Publication 04 February 2025” on the Commission’s library page, have an overview table with eight rows, Article 5(1)(a) to (h) (para 9), and state that the prohibitions apply from 2 February 2025 (para 3).
- High risk. The page lists among high-risk use cases AI systems used for remote biometric identification, emotion recognition and biometric categorisation, and states that from 2 December 2027 high-risk systems will face obligations before they can be put on the market, among them risk assessment and mitigation, high-quality datasets “to minimise risks of discriminatory outcomes”, logging, documentation, human oversight, and robustness, cybersecurity and accuracy.
- Transparency. The page says the transparency rules come into effect in August 2026 and that deep fakes, among other content, should be clearly and visibly labelled; deepfakes summarizes the Commission’s guidelines on those rules.
- Dates. The page states that the Act entered into force on 1 August 2024 and became applicable on 2 August 2026, with exceptions: prohibited practices from 2 February 2025; governance rules and obligations for general-purpose AI models from 2 August 2025; rules for high-risk systems in regulated products (Annex I) with an “extended transition period until 2 August 2028”; and rules for high-risk use cases in sensitive areas (Annex III) “extended to 2 December 2027 as a result of the political agreement on the proposal to simplify the AI Act – ‘AI Omnibus’”. Elsewhere the page says that this legislative proposal “was adopted on 19 November 2025, a political agreement was reached on 7 May 2026 and entered into force on 27 July 2026”. The Omnibus text, which the page links on EUR-Lex, was not read.
In our reading, the dates depend on the instrument. Clinical validation, which summarizes the Act as published on 12 July 2024 and excludes later amendments, reports that in that text the obligations attached to Article 6(1), which makes AI in products such as medical devices high-risk, apply from 2 August 2027; the Commission’s page, as of November 2026, gives 2 August 2028 for the Annex I rules. Radiology AI describes the 2024 text’s high-risk classification of AI in medical devices.
What the guidelines say about vision work
The guidelines state that they are non-binding and that any authoritative interpretation of the Act can only be given by the Court of Justice of the European Union (para 5). Several of their paragraphs concern vision work directly:
- Research. The guidelines explain that the Act does not apply to research, testing or development before a system is placed on the market or put into service (Article 2(8)), nor to systems developed and put into service for the sole purpose of scientific research and development (Article 2(6)) (paras 30–31). Testing in real-world conditions is not covered by the first exclusion; their example is a municipality testing remote biometric identification with volunteers in the streets during a carnival (para 32). Excluded research should still follow recognised ethical and professional standards and applicable Union law, such as data protection law (para 33).
- Face datasets. In the guidelines, the scraping prohibition does not cover facial image databases that are not used to recognise people, such as those used for training or testing where the persons are not identified (para 234), nor systems that harvest facial images to build models generating images of fictitious persons, which could fall under the transparency rules (para 235).
- Labels for representativeness. In the guidelines, labelling or filtering lawfully acquired biometric datasets falls outside the biometric categorisation prohibition, and may be done precisely so that the data equally represent all demographic groups (paras 284–285).
Research Practice
Ethics review: LAION reports that LAION-5B’s datasheet records that the University of Washington’s Human Subjects Division judged the work not to need IRB review and that the NeurIPS ethics review found no serious ethical issues; video surveillance reports that MEVA was recorded under IRB oversight with actors who signed consent forms. Retraction and stewardship: dataset bias and person re-identification report Peng, Mathur and Narayanan’s finding that the retractions of DukeMTMC and MS-Celeb-1M had limited effect, and their call for explicit retractions and checks by program committees. Documentation: datasheets for datasets (also in dataset bias) and annotation cards that disclose who labelled the data and under what terms (image annotation). Impact statements: method papers that state how their methods could be misused (image editing).
Limits of Technical Fixes
NIST SP 1270 (March 2022) observes that organizations often default to overly technical solutions for AI bias, and that mathematical and computational approaches do not adequately capture the societal impact of AI systems (§2.3). It calls for a socio-technical view that considers the system within the social setting where it operates, including whether technology is even a solution to a given task, and names, as one control used in model risk management against techno-solutionism, establishing, documenting and reviewing a system’s anticipated real-world value.
In our reading, the vision articles show the same limit from inside. A blurred licence plate can be re-identified by an attacker who can model the blur and knows the set of possible plates (privacy); when base rates differ, equal error rates across groups make a positive decision mean different things in each group (bias and fairness); a system can pass its test set and still fail a slice or the field (testing and validation). Each fix is worth making, and none settles whether the system should be deployed, which the AI RMF leaves to its MAP go/no-go decision and the joint responsibility of AI actors.
Common Misconceptions
- “If it is legal, it is ethical.” The AI RMF is voluntary, the Commission’s guidelines are non-binding, and the Commission’s page says the Act introduces no rules for AI deemed minimal or no risk. In our reading, legality and ethics are separate questions, and the frameworks address only part of the second.
- “Photos posted publicly are free to use.” The Commission’s guidelines state that posting one’s own face on social media is not consent to its inclusion in a facial recognition database (para 232); dataset bias and the dataset articles document what consent their creators did and did not obtain.
- “Research is exempt from all of this.” The guidelines state that the exclusion for research and development before a system reaches the market (Article 2(8)) does not cover testing in real-world conditions, and that excluded research should still follow ethical and professional standards and data protection law (paras 32–33).
- “Any facial expression analysis is emotion recognition.” In the guidelines, detecting a smile or a driver’s fatigue is not emotion recognition; inferring that someone is happy or angry is (para 249).
- “The AI Act has eight prohibitions and fixed dates.” The count and the dates depend on the document and its date: eight rows in the guidelines’ table of 29 July 2025, nine practices on the Commission’s page as of November 2026, and high-risk dates that differ between the 2024 text, as Clinical Validation reports it, and the Commission’s page after the AI Omnibus.
Where It Is Used
These questions arise wherever vision systems see people, from recognition and surveillance to synthetic media, for which deepfake detection covers labelling and detection, and medical imaging. The quantitative work behind them is in bias and fairness, privacy and testing and validation.
Related
- Video Surveillance and Security
Computer vision applied to video from fixed security and traffic cameras as its research literature describes it; the processing chain from detection to multi-camera tracking, re-identification, attributes, events and search, the PETS, DukeMTMC, CityFlow, AI City Challenge, VIRAT and MEVA benchmarks with their protocols and metrics, documented failure modes, NIST's accuracy and demographic findings, and the privacy concerns stated in official documents.
- Dataset Bias
How the way a dataset's images are chosen, photographed and labelled makes it an unrepresentative sample of the visual world, how cross-dataset tests measure the damage, and what bias means when the images show people, from demographic representation to withdrawn datasets.
- Deepfakes
Realistic synthetic or manipulated images, video and audio of real people; what counts as a deepfake, the method families (face swapping, reenactment, lip sync, attribute editing, full-face synthesis) at survey level, their history, documented uses and harms, consent, legal responses in the EU, the US and the UK as of November 2026, and detection and provenance in outline.
- Face Recognition
Deciding whether two face images show the same person (verification) or which enrolled person a face belongs to (identification, closed- or open-set), with the detect, align, embed and compare pipeline, its benchmarks and evaluation protocols, and NIST's measurements of demographic differentials.
- Testing and Validating Vision Systems
How a vision system is shown to meet its requirements before and after it goes into service, from the test plan and held-out tests to slice analysis, behavioural and metamorphic tests, stress tests at the requirement's limits, regression tests and the field acceptance test, with an inspection example in which the aggregate passes, one slice fails and a label-free test points to it.
- Clinical Validation and Regulation of Medical Imaging AI
How the claims made for a medical image analysis model are tested, from internal, temporal and external validation through reader studies, prospective studies and randomized trials, with the reporting guidelines that describe such studies, the statistics of rare findings, the regulatory pathways in outline, and monitoring after deployment.
References
- National Institute of Standards and Technology (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST AI 100-1, January 2023.
- Schwartz, R., Vassilev, A., Greene, K., Perine, L., Burt, A. & Hall, P. (2022). Towards a Standard for Identifying and Managing Bias in Artificial Intelligence. NIST Special Publication 1270, National Institute of Standards and Technology.
- European Commission (2025). Commission Guidelines on prohibited artificial intelligence practices established by Regulation (EU) 2024/1689 (AI Act). Communication from the Commission, C(2025) 5052 final, dated Brussels, 29 July 2025; library page: “Publication 04 February 2025” (accessed 14 November 2026).
- European Commission. AI Act. Shaping Europe’s digital future, policy page, last updated 19 October 2026 (accessed 14 November 2026).
- Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union, L series, 12 July 2024 (not read; cited through the Commission’s guidelines).