Open model licences, datasets and deployment obligations

Learn in your browser · Chapter 01 of 06

Many releases are called open. Pin down what was actually released, and what the Open Source AI Definition asks for.

An open-weight model has downloadable weights under some licence. Open source is a stricter claim. The Open Source Initiative's Open Source AI Definition 1.0 (OSAID) requires freedoms to use, study, modify and share. The first reads: "Use the system for any purpose and without having to ask for permission." It also requires the preferred form to make modifications:

Scroll sideways to see every column.

The OSI's FAQ adds that it validates legal documents, not individual AI systems. A release is a bundle: weights, code, datasets, hosting terms and acceptable-use policies can each carry different terms, so record each one.

Learn in your browser · Chapter 02 of 06

Model licences fall into three broad families. Whatever the family, ten questions show what you may do.

Scroll sideways to see every column.

Apache-2.0 grants copyright and patent licences. Redistributors must include the licence, mark changed files, keep notices and pass on any NOTICE file's contents (section 4). The patent licence ends for anyone who sues claiming the work infringes a patent (section 3). MIT asks that its notices travel with copies.

RAIL licences carry behavioural-use restrictions that must bind downstream users and derivatives. The RAIL FAQ says OpenRAIL licences are not open source under the OSI's definition.

Custom licences vary most. A custom community licence may combine a user-count threshold above which you need a separate licence, a required credit display and notice file, a naming rule for derived models, an incorporated acceptable-use policy, termination on breach and a foreign governing law.

Scroll sideways to see every column.

Learn in your browser · Chapter 03 of 06

A dataset carries two layers of rights: the licence on the collection, and the rights in each item inside it. Check both.

Scroll sideways to see every column.

A Creative Commons deed "is not a license and has no legal value", so read the legal code. Creative Commons' FAQ says the NC definition "does not turn on the type of user".

Licence versus items. The deeds warn that publicity, privacy or moral rights may still limit use, and Creative Commons has not verified the copyright status of works marked CC0. A dataset labelled CC BY can hold items its publisher could not license. Scraped data adds the platform's terms, a contract. Creative Commons' September 2026 guidance adds that copyright exceptions may permit some AI uses whatever a licence says.

Personal data brings data protection law whatever the licence. For training generative AI on web-scraped personal data, the ICO says legitimate interests is the sole available lawful basis on current practices, if its three-part test is passed. See Privacy and data minimisation in AI applications.

Learn in your browser · Chapter 04 of 06

Documentation is your evidence. Learn what good records contain, then keep your own in a standard form.

Model cards (Mitchell and colleagues, 2018) give intended uses, evaluation across relevant groups, and limitations. Datasheets for datasets (Gebru and colleagues, 2018) record motivation, composition, collection and recommended uses. On the Hugging Face hub (a documented practice, not a recommendation) a card's metadata can name the licence, datasets and a base_model for fine-tuned versions. Treat cards as dated claims, not proof.

The NCSC's secure AI guidelines ask you to document training data sources, limitations and cryptographic hashes or signatures, using model cards, data cards and software bills of materials (SBOMs), which CISA calls a nested inventory of ingredients. Two open standards cover AI:

Scroll sideways to see every column.

Copy SPDX's split: the declared licence is the one found in the artefact; the concluded licence is the one the record's author concluded applies. Identifiers such as CC-BY-NC-4.0 name listed licences; LicenseRef- marks others.

Learn in your browser · Chapter 05 of 06

Turn the questions into a small tool that reads an inventory, flags problems and drafts a NOTICE file. Everything is invented; nothing is downloaded.

In a new folder, save this as inventory.json: three invented models with planted mistakes. Sources use example.org, which IANA keeps for documentation.

[
  {"model": "fern-summarise", "version": "1.2", "licence": "Apache-2.0",
   "source": "example.org/fern-summarise",
   "sha256": "5f40da6a3889713715188839b3f2901be2b0ff831bb5bf8d51f6fc4e6625b08e",
   "dataset": "civic-letters", "dataset_licence": "CC-BY-4.0",
   "intended_use": ["summarisation"], "commercial": false,
   "redistribute": true, "above_threshold": false,
   "notice": "Copyright 2025 Fernwood Labs (invented)"},
  {"model": "harbour-triage", "version": "0.9", "licence": "LicenseRef-Harbour-RAIL-1.0",
   "source": "example.org/harbour-triage", "sha256": "",
   "dataset": "clinic-notes", "dataset_licence": "CC-BY-NC-4.0",
   "intended_use": ["triage", "medical-advice"], "commercial": true,
   "redistribute": false, "above_threshold": false, "notice": ""},
  {"model": "kestrel-chat", "version": "3.0", "licence": "LicenseRef-Kestrel-Community-2.0",
   "source": "example.org/kestrel-chat",
   "sha256": "40c74848309fe8d05b570ea39700045a580bfeafb791a54a193bc9ae850a7634",
   "dataset": "forum-scrape", "dataset_licence": "",
   "intended_use": ["chat"], "commercial": false,
   "redistribute": true, "above_threshold": null, "notice": ""}
]

The tool holds simplified summaries. STOP means do not deploy, FIX means an incomplete record, DUTY is something you must do. Save it as licence_check.py beside the inventory.

"""licence_check.py: report licence problems; --write drafts a NOTICE file."""
import json
import re
import sys
from pathlib import Path

# Simplified summaries. Both LicenseRef- licences are INVENTED.
MODELS = {"Apache-2.0": {}, "MIT": {},
          "LicenseRef-Harbour-RAIL-1.0": {"restricted": ["medical-advice"], "flow_down": True},
          "LicenseRef-Kestrel-Community-2.0": {"threshold": True}}
DATA = {"CC-BY-4.0": True, "CC0-1.0": True, "CC-BY-NC-4.0": False}  # commercial use allowed?


def check(e):
    lic = MODELS.get(e["licence"])
    issues = [] if lic is not None else [("STOP", f"unknown licence '{e['licence']}'")]
    lic = lic or {}
    for use in e["intended_use"]:
        if use in lic.get("restricted", []):
            issues.append(("STOP", f"licence restricts '{use}'"))
    if lic.get("threshold") and e["above_threshold"] is not False:
        issues.append(("STOP", "user threshold passed or unknown"))
    if not re.fullmatch(r"[0-9a-f]{64}", e["sha256"]):
        issues.append(("FIX", "sha256 missing or malformed"))
    if e["redistribute"] and not e["notice"]:
        issues.append(("FIX", "no attribution text for NOTICE"))
    if e["dataset_licence"] not in DATA:
        issues.append(("STOP", f"unknown dataset terms '{e['dataset_licence']}'"))
    elif e["commercial"] and not DATA[e["dataset_licence"]]:
        issues.append(("STOP", "dataset licence forbids commercial use"))
    if lic.get("flow_down"):
        issues.append(("DUTY", "pass use restrictions on to users"))
    return issues


inventory = Path(sys.argv[1])
totals, notice = {}, ["NOTICE (practice file)"]
for e in json.loads(inventory.read_text(encoding="utf-8")):
    print(e["model"], e["version"], e["licence"])
    for level, text in check(e) or [("ok", "no issues found")]:
        print(f"  {level:<4} {text}")
        totals[level] = totals.get(level, 0) + 1
    if e["redistribute"]:
        notice += [f"{e['model']} {e['version']} ({e['licence']})",
                   "  " + (e["notice"] or "ATTRIBUTION MISSING: fix before release")]
print("Totals:", totals)
out = Path("practice") / f"NOTICE-{inventory.stem}.txt"
if "--write" not in sys.argv:
    print(f"Dry run: {out.name} not written. Add --write.")
elif out.exists():
    print(f"Skipped: {out.name} already exists")
else:
    out.parent.mkdir(exist_ok=True)
    out.write_text("\n".join(notice) + "\n", encoding="utf-8")
    print("Wrote", out.as_posix())

Only false passes a threshold check: null means nobody checked. A missing field stops Python with a KeyError. Nothing is written without --write, or overwritten: to redo a NOTICE file, delete the old one. Run the dry run (python3 on macOS and Linux):

python licence_check.py inventory.json
fern-summarise 1.2 Apache-2.0
  ok   no issues found
harbour-triage 0.9 LicenseRef-Harbour-RAIL-1.0
  STOP licence restricts 'medical-advice'
  FIX  sha256 missing or malformed
  STOP dataset licence forbids commercial use
  DUTY pass use restrictions on to users
kestrel-chat 3.0 LicenseRef-Kestrel-Community-2.0
  STOP user threshold passed or unknown
  FIX  no attribution text for NOTICE
  STOP unknown dataset terms ''
Totals: {'ok': 1, 'STOP': 4, 'FIX': 2, 'DUTY': 1}
Dry run: NOTICE-inventory.txt not written. Add --write.

All six planted problems are caught. With --write the last line becomes Wrote practice/NOTICE-inventory.txt, and the file shows the gap:

NOTICE (practice file)
fern-summarise 1.2 (Apache-2.0)
  Copyright 2025 Fernwood Labs (invented)
kestrel-chat 3.0 (LicenseRef-Kestrel-Community-2.0)
  ATTRIBUTION MISSING: fix before release

Deployment obligations. The tool drafts; people decide. Before release, check:

• Notices: licence copies, NOTICE contents and marked changes (Apache-2.0 section 4); credit lines or name prefixes custom licences require.
• Flow-down: use restrictions and acceptable-use policies reach your users through your own terms.
• Transparency: GOV.UK says the Algorithmic Transparency Recording Standard is mandatory for in-scope tools in government departments and in arm's-length bodies delivering public or frontline services or dealing directly with the public. Records ask about third-party suppliers and models.
• Logs and incidents: the UK's voluntary AI cyber security code asks operators to log system and user actions, plan for incidents and tell end-users before model updates. The UK GDPR requires certain personal data breaches to be reported to the ICO within 72 hours, where feasible.

Learn in your browser · Chapter 06 of 06

A licence covers a specific file. Check that the file you hold is that file, and that opening it cannot run someone else's code.

A SHA-256 hash is a 64-character fingerprint of a file's exact bytes. The UK's voluntary AI cyber security code says developers shall release hashes for model components. A match proves your copy equals the published value, not who published it, so take the value from the publisher's own page and verify signatures where offered.

Save verify.py beside the inventory. It checks stand-in text files, not models; --make creates them. hashlib.file_digest needs Python 3.11 or later.

"""verify.py: compare stand-in files with the inventory's SHA-256 values."""
import hashlib
import json
import sys
from pathlib import Path

folder = Path("stand-ins")
if "--make" in sys.argv:  # fails if the folder exists
    folder.mkdir()
    (folder / "fern-summarise.txt").write_bytes(b"stand-in for model weights\n")
    (folder / "kestrel-chat.txt").write_bytes(b"stand-in, altered in transit\n")

for e in json.loads(Path("inventory.json").read_text(encoding="utf-8")):
    path = folder / f"{e['model']}.txt"
    if not e["sha256"]:
        print(path.name, "no hash recorded, cannot verify")
        continue
    with open(path, "rb") as f:
        actual = hashlib.file_digest(f, "sha256").hexdigest()
    print(path.name, "MATCH" if actual == e["sha256"] else "MISMATCH, do not use")

python verify.py --make
fern-summarise.txt MATCH
harbour-triage.txt no hash recorded, cannot verify
kestrel-chat.txt MISMATCH, do not use

Both planted problems show. A mismatch means stop and fetch again from the official source.

Pickled files. Python's documentation warns: "The pickle module is not secure. Only unpickle data you trust." A pickle can name a function to call while loading. This harmless demonstration reads the steps without running them, then loads:

"""pickle_demo.py: harmless proof that loading a pickle can run code."""
import pickle
import pickletools


class Surprise:
    def __reduce__(self):
        return (print, ("This ran while the file was loading.",))


data = pickle.dumps(Surprise(), protocol=4)
pickletools.dis(pickletools.optimize(data))  # reads the steps, runs nothing
pickle.loads(data)  # loading runs them

python pickle_demo.py
    0: \x80 PROTO      4
    2: \x95 FRAME      59
   11: \x8c SHORT_BINUNICODE 'builtins'
   21: \x8c SHORT_BINUNICODE 'print'
   28: \x93 STACK_GLOBAL
   29: \x8c SHORT_BINUNICODE 'This ran while the file was loading.'
   67: \x85 TUPLE1
   68: R    REDUCE
   69: .    STOP
highest protocol among opcodes = 4
This ran while the file was loading.

STACK_GLOBAL finds builtins.print and REDUCE calls it, so the last line printed during loading. python -m pickletools disassembles a file without loading it. Safer formats hold data, not instructions: safetensors is a header length, a JSON header and raw tensor bytes, designed so loading runs no arbitrary code. Loaders and shipped code still need checking.