Lake Formation Governance: Permissions, Row/Column Security, and Data Sharing

Lessons 01 to 11 built the machine: buckets, formats, Iceberg tables, crawlers, jobs, a query engine, a warehouse and an orchestrator. Every one of those layers produced a new place where somebody had to answer the same question; who may see this? This lesson answers it once, centrally, with AWS Lake Formation. The service is not an IAM replacement and not a catalog; it is an RDBMS-shaped permission layer that sits on the Glue Data Catalog and vends scoped temporary credentials to integrated engines, so that one GRANT statement can mean something at database, table, column, row and cell granularity.

The teaching rule for this lesson: Lake Formation decides visibility, IAM decides reachability, and the engine does the actual filtering. Confuse those three and every symptom in this lesson becomes a mystery.

In this lesson you will:

• diagnose what Lake Formation actually replaces, the IAM-plus-S3 policy sprawl, and what it deliberately does not replace;
• separate LF-registered locations from hybrid access mode, and predict which permission plane answers a given request;
• climb the permission granularity ladder (database → table → column → row filter → cell) and name the mechanism at each rung;
• build the worked analyst policy: SELECT on gold.sales, minus salary, rows limited to region = 'eu', as one cell-level data filter;
• apply LF-Tags and TBAC: tag grants, inheritance, and the AND-across-keys / OR-within-a-key matching rules;
• share cross-account databases through AWS RAM, with the external-engine toggle, trusted callers, managed access and data-lake-administrator rules;
• place Athena, Redshift Spectrum, QuickSight, Glue and EMR on an engine support matrix and say what breaks in each cell;
• remove the IAMAllowedPrincipals default and execute the documented six-step migration;
• read the permission-check decision tree and write real GrantPermissions calls in JSON and CLI;
• study three named AWS customer cases (Integral Ad Science, FanDuel, Morningstar), a sourced October 2026 update box, nine worked examples, five diagrams, three interactive checks and twelve exam-style questions.

---

1. What Lake Formation replaces: the IAM and S3 sprawl

1.1 Three surfaces, one table, no rows

Before Lake Formation, a governed lake on AWS was assembled from three overlapping policy surfaces, and AWS said so at general availability on 2019-08-08:

"Previously you had to use separate policies to secure data and metadata access, and these policies only allowed table-level access. Now you can give access to each user, from a central location, only to the columns they need to use.", AWS News Blog, AWS Lake Formation, Now Generally Available (2019-08-08)

Lake Formation adds a fourth surface that is deliberately shaped like a relational database grant system:

"Lake Formation provides a relational database management system (RDBMS) permissions model to grant or revoke access to catalogs, databases, tables, and columns in the Data Catalog with underlying data in Amazon S3.": LF Developer Guide, Onboarding to Lake Formation permissions (accessed 2026-10-11)

And it augments rather than replaces IAM:

"Lake Formation provides its own permissions model that augments the IAM permissions model.": LF Developer Guide, Overview of data filtering (accessed 2026-10-11)

The division of labour is therefore fixed for the rest of the lesson: IAM authenticates and gates the control plane (GrantPermissions, GetDataAccess, glue:*), Lake Formation gates the data plane by deciding what metadata and what scoped temporary credentials an integrated engine receives, and the engine performs the row/column/cell filtering itself.

Two facts fall straight out of that diagram and both are examinable:

1. Registration is the only bridge between the planes. "Lake Formation credential vending is not available, even when opted in, unless the data location is registered with Lake Formation." (LF Developer Guide, Hybrid access workflow, accessed 2026-10-11.)
2. Lake Formation does not lock your bucket. "The Lake Formation permissions model doesn't prevent access to Amazon S3 locations through the Amazon S3 API or console if you have access to them through IAM or Amazon S3 policies. You can attach IAM policies to principals to block this access." (LF Developer Guide, Underlying data access control, accessed 2026-10-11.)

1.2 Worked example E1: counting the sprawl, then counting the replacement

Reading: Lake Formation's first win over raw IAM is not fewer statements: at named-resource granularity you write the same number. Its first win is centralisation plus auditability (one ListPermissions instead of three policy families), and its second win, which arrives in Section 5, is collapsing the count with tags. The cost column is the surprise: the governance layer itself is free, so the only budget question is who maintains it.

• 📚 Did you know? Lake Formation billing has exactly three billable items as of Oct 2026, and none of them is a permission. The Storage API, the standalone server-side filtering engine used when no AWS analytics service processes the data, bills "for the number of bytes scanned by the Storage API, rounded to the next megabyte, with a 10MB minimum"; Governed Tables bills per file tracked, per metadata API call and per byte processed by the storage optimizer; and the storage optimizer itself bills per byte. Grants, revokes, LF-Tags, cross-account sharing and enforcement by Athena, Redshift, Glue and EMR are all $0 (Lake Formation pricing page, accessed 2026-10-11).

---

2. LF-registered versus hybrid: the two permission modes

A Data Catalog object has two orthogonal questions, and most Lake Formation confusion comes from answering only one of them:

2.1 Registration: what the role is for

"When you register a location, you specify an IAM role that grants read/write permissions on that location. Lake Formation assumes that role when supplying temporary credentials to integrated AWS services.": LF Developer Guide, Set up AWS Lake Formation (accessed 2026-10-11)

Practical consequences:

• the registration role, not the caller, holds the durable s3:* permissions;
• callers additionally need the IAM permission lakeformation:GetDataAccess, and "you must set the Resource parameter to *": any other resource value is unsupported;
• registering a prefix covers every table under that path at once, because "multiple tables can share an S3 prefix. When you register a location, all tables under that path get credential vending at once" (AWS-published Lake Formation best-practices guide, accessed 2026-10-11).

2.2 Hybrid access mode: two pathways, opt-in principals

"AWS Lake Formation hybrid access mode supports two permission pathways to the same AWS Glue Data Catalog objects." … "The hybrid access mode enforces only CREATE_TABLE, CREATE_PARTITION, UPDATE_TABLE permissions by default." … "opted in principals will require both Lake Formation permissions and IAM permissions to access the data, while non-opted-in principals will continue to access data using only IAM permissions.": LF Developer Guide, Hybrid access mode (accessed 2026-10-11)

That last sentence is the whole mode: hybrid is a per-principal switch, not a per-account switch, and it is the reversible migration tool AWS recommends:

"To ease the transition of data lake permissions from an IAM and Amazon S3 model to Lake Formation permissions, we recommend you to use hybrid access mode for Data Catalog.": LF Developer Guide, Set up AWS Lake Formation (accessed 2026-10-11)

2.3 Worked example E2: taking one role into hybrid mode

Checks: (a) the opted-in analyst with an IAM policy but no LF grant now fails, proof the double gate is live; (b) the non-opted-in role still reads with IAM only, proof the mode is per principal; (c) IsRegisteredWithLakeFormation on the table is true for the opted-in principal's view and unchanged for everyone else.

• 📚 Did you know? The account-level knobs that keep a new database on the old plane have exact console names: "Use only IAM access control for new databases" and "Use only IAM access control for new tables in new databases", backed in the API by CreateDatabaseDefaultPermissions and TableDefaultPermissions, whose shipped default is [{"Principal":{"DataLakePrincipalIdentifier":"IAM_ALLOWED_PRINCIPALS"},"Permissions":["ALL"]}] (LF Developer Guide, Changing the default settings for your data lake and DataLakeSettings API, accessed 2026-10-11). AWS deliberately does not name this state "IAMOnly" anywhere: teach the documented names.

---

3. The permission granularity ladder

3.1 The vocabulary

Resource kinds you can grant on: Catalog, Database, Table, View, DataLocation (only DATA_LOCATION_ACCESS applies), LF_Tag, LF_TagPolicy / LFTagExpression.

The permissions enum, as returned by ListPermissions (LF API reference, accessed 2026-10-11):

Three rules about that list:

• SELECT, INSERT and DELETE are both Data Catalog permissions and data-access permissions; they are the ones that reach S3.
• Super is additive and survives partial revocation: "when you revoke Super, the SELECT and INSERT permissions remain" (LF Developer Guide, Access control for metadata, accessed 2026-10-11).
• Granting on all tables in a database implicitly grants DESCRIBE on that database, but "this automatic DESCRIBE permission grant doesn't apply when using attribute-based access control (ABAC)".

3.2 The ladder, drawn

Everything below TABLE is expressed through two mechanisms only: a column filter on the grant itself (TableWithColumns) or a data filter object (DataCellsFilter). Choose between them with one question: do I need rows too?

3.3 Worked example E3: the four-grant ladder on one table

Numbers as of Oct 2026: a data filter needs SELECT with the grant option on all table columns to create; there is no cap on filters per table but a limit of 100 data filters per principal per a single table; and the row expression must be under 2,048 characters (LF Developer Guide, Data filtering limitations and PartiQL support, accessed 2026-10-11).

• 📚 Did you know? Three ceilings are worth memorising because they are all small: 30 data lake administrators per account by default, 1,000 saved LF-Tag expressions per account, and 100 data filters per principal per table (LF Developer Guide, Set up AWS Lake Formation, Managing LF-Tag expressions and Data filtering limitations, all accessed 2026-10-11). A fourth, softer one: AWS's own tag-based worked example reaches 17 effective permissions from just 5 tag assignments plus 8 tag grants.

---

4. Row-level and column-level security: the worked analyst policy

4.1 The combinatorics: three controls, one object type

"Lake Formation uses data filtering to achieve column-level security, row-level security, and cell-level security." … "When you specify the 'all columns' wildcard and provide a row filter expression, you are establishing row-level security (row filtering) only." … "When you include or exclude specific columns and nested columns, and specify 'all rows' … you are establishing column-level security." … "When you include or exclude specific columns and also provide a row filter expression, you are establishing cell-level security.": LF Developer Guide, Data filtering and cell-level security (accessed 2026-10-11)

Two clarifications that save real debugging time:

• Column filtering is not masking. Excluding salary removes it from results and from that principal's catalog metadata. It never returns ***. If the requirement is "show last four digits only", that is a view, an application-layer transform, or Amazon Redshift's dynamic data masking: not Lake Formation.
• Availability: "Cell-level security is available in all regions when using Athena engine version 3 or Amazon Redshift Spectrum." (LF Developer Guide, Data filtering limitations, accessed 2026-10-11.)

4.2 The row-filter language: PartiQL subset, with hard edges

4.3 Worked example E4, the analyst policy: gold.sales minus salary, rows = region='eu'

This is cell-level security in one data filter object, plus the visibility grant that makes it usable.

Four learner checks, and what each one proves:

Contrast (cheaper, narrower): granting TableWithColumns with ExcludedColumnNames = ["salary"] gets you the columns and no rows: a legitimate answer when the requirement is only "hide PII", and the pattern FanDuel used in Case B below. Adding region = 'eu' is what forces you into a data filter.

---

5. LF-Tags and tag-based access control (LF-TBAC)

5.1 What a tag is

"Each LF-Tag is a key-value pair, such as department=sales or classification=restricted. A key can have multiple defined values.": LF Developer Guide, Tag-based access control (LF-TBAC) (accessed 2026-10-11)

Tags attach to databases, tables and columns. They must be predefined (a key and its values are created with CreateLFTag before anything can carry them), and they are the only mechanism in this lesson that lets you grant on an attribute rather than on a named object.

5.2 Inheritance and the matching rules

"Tables inherit LF-Tags from databases and columns inherit LF-Tags from tables. Inherited values can be overridden" … "the tag keys are combined using the AND operation, while the values are combined using the OR operation.": LF Developer Guide, Tag-based access control (LF-TBAC) (accessed 2026-10-11)

The two combination rules, stated once and tested forever:

• Different keys in one grant are AND-ed: owner = data-platform AND class = internal matches only resources carrying both.
• Multiple values of the same key are OR-ed: class = internal OR class = restricted matches either.
• Example from the docs: the tag expression content_type:Sales AND location:US "represents resources related to sales data in the US"; saved LF-Tag expressions are capped at 1,000 per account.

5.3 Three things TBAC changes about granting

1. You grant on the tag, not the table. The principal's grant resource is an LFTagPolicy (or an LFTagExpression), so tables created later, including crawler-created tables, are covered automatically once they inherit the tag.
2. You can grant the ability to tag. ASSOCIATE lets a principal attach a tag; DESCRIBE lets them see it; GRANT_WITH_LF_TAG_EXPRESSION lets them use it as a grant resource. This is the delegation primitive behind data meshes.
3. Permissions from all methods are unioned. AWS's own migration advice: "(Recommended) After verifying that principals have access to the correct resources through the LF-TBAC method, revoke permissions that were granted by using the named resource method." Verify first, revoke second: never the reverse.

5.4 Worked example E5: rebuilding the IAS compression with three tags

Reading: the tag route's cost is flat in the number of tables and linear in the number of policies; the named route is linear in tables × principals. IAS reported the real-world version of this: "With Lake Formation tag-based access controls, IAS reduced hundreds of permission rules down to precisely two rules" (AWS Big Data Blog, 2021-09-23). Note where the exception list lands: row filters are exactly the feature that cannot be tag-driven, so a TBAC estate always carries a short, deliberate named-resource tail.

Hover a bar pair: named-resource grants scale as principals × tables (derived: 4 × 6 = 24 up to 4 × 36 = 144), while the tag route stays at 6 assignments + 4 grants = 10 regardless of table count (derived from the IAS shape of two keys applied at database level). The crossover is immediate, the tag route is already smaller at six tables, which is why AWS's tag-based example is quoted as operational simplification rather than as a security feature.

---

6. Data sharing: cross-account databases, external engines, trusted callers

6.1 Cross-account databases go through AWS RAM

"Lake Formation uses the AWS Resource Access Manager (AWS RAM) service to share the resource. If the grantee account is in the same organization as the grantor account, the shared resource is available immediately; if the grantee account is not in the same organization, AWS RAM sends an invitation to the grantee account to accept or reject the resource grant.": LF Developer Guide, Cross-account data sharing (accessed 2026-10-11)

Two grant methods exist, and AWS recommends one of them by name: Lake Formation tag-based access control (LF-TBAC) (recommended) versus named resources. The blog that documents the trade-off puts it plainly: TBAC "is recommended when many resources and entities are involved."

Prerequisites that are hard failures, not advice (LF Developer Guide, Cross-account prerequisites, accessed 2026-10-11):

1. Revoke all Lake Formation permissions from IAMAllowedPrincipals on the resource first: otherwise Lake Formation throws AccessDeniedException.
2. The grantor must hold the AWS managed policy AWSLakeFormationCrossAccountManager.
3. At least one user in the consumer account must be a data lake administrator to view shared resources; "other principals can't access shared resources until the data lake administrator grants them permissions."
4. Cross-account grants must carry the grant option, because the consumer-side admin re-grants locally. "The principal who receives the resource share cannot share the resource with other principals."

Cross-account version settings (DataLakeSettings.Parameters.CROSS_ACCOUNT_VERSION) decide what can be shared: direct grants to an individual IAM principal in another account need version ≥ 3; hybrid-mode shares need ≥ 4; version 5 "enhances cross-account resource sharing enabling you to share unlimited number of tables to another account, eliminating previous resource association limits per resource type", is available "through the Lake Formation console or API", and downgrade is not supported once applied (AWS What's New, 2026-02-11; LF Developer Guide, accessed 2026-10-11).

6.2 Worked example E6: sharing gold.sales to an external account

Checks: (a) a grant attempted while IAMAllowedPrincipals still holds Super fails with AccessDeniedException; (b) a grant without --permissions-with-grant-option leaves the consumer admin unable to grant their own analysts anything; (c) granting only on the link leaves the table invisible to queries: link grant ≠ target grant.

6.3 Trusted callers and the external-engine toggle

Lake Formation vends temporary credentials only to trusted callers. "Each of these AWS services is a trusted caller to Lake Formation, and Lake Formation provides access to data stored in Amazon S3 through temporary credentials": the integrated set being Athena, Redshift Spectrum, EMR and Glue (LF Developer Guide, Working with services, accessed 2026-10-11).

Anything else is opt-in, and the opt-in is a three-part switch in DataLakeSettings:

"If you intend to analyze … using third-party query engines, you must opt in to allow external engines to access data managed by Lake Formation.": LF Developer Guide, Set up AWS Lake Formation (accessed 2026-10-11)

A documentation discrepancy to be honest about: the DataLakeSettings API reference shows an example session tag key of LakeFormationTrustedCaller, while the third-party-engine guide and the custom-application blog both specify LakeFormationAuthorizedCaller. Two sources say the latter, so use it, but do not treat either name as uncontested.

6.4 Managed access and the administrator roles

Two asymmetries are worth internalising:

• Admin added later = zero implicit data access. "data lake administrators added after database creation can grant permissions but don't automatically have data access permissions such as SELECT or DESCRIBE. Administrators who create databases receive SUPER permissions on those databases."

• Managed access is expressed as the grant option. The pattern, a platform team holding PermissionsWithGrantOption and domain teams re-granting within a scope, is how "grantable" self-service works without handing over the whole catalog.

• 📚 Did you know? The registration role and the service-linked role are a real design choice: Lake Formation ships a service-linked role (ServiceRoleForLakeFormationDataAccess) for zero-setup registration, but AWS's best-practices guide recommends registering with a customer-managed role when you expect to edit or scope that role later. Either way the caller still needs lakeformation:GetDataAccess with Resource: "*", and adding it is "a no-op when no locations are registered, so it is safe (and recommended) to add upfront" (AWS-published Lake Formation best-practices guide, accessed 2026-10-11).

---

7. Engine support matrix: who actually enforces what

"the actual filtering of columns in query responses is the responsibility of the integrated service. Lake Formation clients that support column-level filtering, including Amazon Athena, Amazon Redshift Spectrum, and Amazon EMR filter the data based on the column permissions registered with Lake Formation": LF Developer Guide, Limitations (accessed 2026-10-11)

7.1 Worked example E7: placing eight engines on the matrix

Rule of thumb derived from the matrix: Lake Formation publishes filtered metadata and scoped credentials; the engine prunes the response. Any engine outside that list either refuses the query or, worse, returns unfiltered data if you granted the external-engine opt-in.

• 📚 Did you know? QuickSight's integration has an edition split that is easy to miss: Enterprise edition lets you "grant fine-grained access control (FGAC) permissions to individual QuickSight users and groups", while Standard edition grants go to IAM roles, with the default service role named aws-quicksight-service-role-v0. Once the source bucket is registered, "Amazon S3 access settings in QuickSight are ignored, including scope-down IAM policies for users and groups": Lake Formation becomes the single source of truth (QuickSight and AWS Big Data Blog documentation, accessed 2026-10-11).

---

8. The IAMAllowedPrincipals default, and the migration off it

8.1 Why your grants appear to do nothing

"Lake Formation sets Super permission on all databases and tables in the Data Catalog to a group called IAMAllowedPrincipals by default. If this group permission exists … all principals in your account will have access to the resource through the IAM principal policies for AWS Glue.": LF Developer Guide, Permissions reference (accessed 2026-10-11)

And the consequence, stated just as directly:

"These settings effectively cause access to Data Catalog resources and Amazon S3 locations to be controlled solely by AWS Identity and Access Management (IAM) policies. Individual Lake Formation permissions are not in effect.": LF Developer Guide, Changing the default settings for your data lake (accessed 2026-10-11)

So the default state of a brand-new or long-unmodified lake is: Lake Formation installed, but switched off. Your shiny grants are recorded, audited, and ignored.

8.2 Worked example E8: the documented migration, rehearsed

---

9. The permission-check flow: a decision tree

This mirrors Lake Formation's documented hybrid-access workflow and is the fastest way to turn "it doesn't work" into a single check.

Read it as a diagnostic order: registration first, IAMAllowedPrincipals second, grant method third, entitlement type fourth. Skipping straight to "is my SQL right?" is the most expensive way to debug Lake Formation.

---

10. Worked Grant API and CLI sketch

10.1 The four call shapes you will actually write

10.2 Worked example E9: reading back a grant correctly

Design note: because there is no deny statement in Lake Formation, least privilege means grant nothing by default and treat every new grant as a widening event that must be re-audited. ColumnWildcard.ExcludedColumnNames only ever narrows the entitlement it is written into.

---

2026 Updates (as of October 2026)

• 📚 Did you know? Two of these updates are quietly load-bearing for this lesson. ABAC (attribute-based access control via tags) was formally documented on 2025-04-17, which is why tag-expression grants now appear alongside named-resource grants in the same permission list; and the Governed Tables sunset means any "governed ACID sharing" requirement should now be answered with Iceberg plus Lake Formation grants, not with a retired table type.

---

Real-World Case Studies

AWS publishes what these permission models look like at production scale. Every figure below is customer- or AWS-claimed and unaudited, with the source and date named so you can check it: the examinable point is the pattern, not the marketing.

Case A, Integral Ad Science: hundreds of permission rules collapsed to two

The examinable pattern: TBAC's real payoff is operational. An ontology of a handful of inherited tags replaces an unmanageable matrix of per-table grants: the Section 5.4 arithmetic in production. Note also what IAS did not skip: column-level controls plus tagging carried the compliance answer, and the tag grant still had to coexist with whatever row-level exceptions the business needed.

Case B, FanDuel: column grants instead of copy-based masking

The examinable pattern: column-level grants kill copy-based masking. The requirement "hide PII from general analysts" is satisfied by one TableWithColumns exclusion on one cataloged table: no second copy of the data, no per-source tagging ritual. This is Section 4's cheaper alternative, and it is why FanDuel never needed row filters for that particular problem.

Case C, Morningstar: one tag ontology, two enforcement planes

The examinable pattern: the tag ontology is portable, the enforcement is not. Redshift supports resource-based entitlements but not tag-based ones, so a team that wants one governance model must build the translation; which is precisely the boundary the engine matrix in Section 7 is drawing.

• 📚 Did you know? All three cases date from 2020 to 2023 and should be presented that way: FanDuel's 8.5 million customers and IAS's "hundreds → two rules" are snapshots of those deployments at the time AWS published them, not current states of those platforms. Their architecture claims, column grants, inherited tags, one workgroup per business unit, remain the documented patterns; their numbers are historical.

---

Practice Questions