Apache Ranger: Policy Model


Madhan Neethiraj                                                                                

2022-March-07



Introduction

Policy Model

    Resources

    Permissions

    Users, Groups, Roles

    Security Zones

    Allow, Deny, Exceptions

    Wildcards, macros, variables in resource names

    Policy validity schedule

Attribute-based access control

Resource-based access control

Tag-based access control

Data masking

Row-filter

Access audit logs

References


Introduction

Apache Ranger is an extensible framework that enables enterprises to adopt a consistent approach to authorize access to their resources across multiple services/applications/cloud. Apache Ranger framework also enables enterprises to collect audit logs of access to their resources, to help meet various compliance requirements.


Apache Ranger is a central part of security in many large deployments in enterprises across various domains like finance, retail, insurance, healthcare, services. Apache Ranger has out-of-the box support for a large number of popular services and many more services are supported by commercial vendors. Apache Ranger is highly optimized for performance, adds negligible overhead in authorizing access to resources. It has been very well proven in very high throughput services like Apache Kafka, Apache HBase which perform thousands of authorizations per second.


Apache Ranger provides an intuitive web user interface to manage authorization policies and audit logs for access to resources across a large number of services. Apache Ranger also provides REST, Python, Java APIs for programmatic integration with tools used by enterprises. Open framework provided by Apache Ranger enables enterprises to extend Apache Ranger authorization to their own applications and services as well.


Here are few key points that make Apache Ranger a compelling option for enterprises looking to standardize authorization of access to their resources:

  • out-of-the-box support for more than a dozen popular services like Apache Hive, Apache HBase, Apache Kafka, Apache Solr, Elasticsearch, Apache NiFi and Presto

  • support for services like Amazon EMR, AWS S3, ADLS-Gen2, GCS, Snowflake, Google BigQuery, Trino, Dremio, Starburst, Apache Impala, Postgres, MS-SQL and Amazon Redshift by commercial vendors

  • policies for access authorization, row-filters, data masking

  • resource-based, classification-based policies, role-based, attribute-based policies

  • delegated administration, deny and exceptions in policies, custom conditions

  • centralized audit logs of accesses to enterprise resources across multiple services, interactive user interface to view audit logs of accesses

  • intuitive policy management UI

  • Java, Python, REST APIs for programmatic integration for policy management

  • open framework which enables enterprises to extend Apache Ranger authorization to their own applications and services


Policy Model

At the core of Apache Ranger authorization is its policy model. We will go through key aspects of the Apache Ranger policy model in this section.


Resources

A resource is a fundamental element in the Apache Ranger policy model. Apache Ranger enables policies to authorize access to resources. In this context, a resource is anything whose access needs to be authorized, like a file/path, database, table, column, topic; but can also be a service – like Apache Knox topology. Apache Ranger policy model captures details of resources of a service in a declarative way – details like hierarchy, case-sensitivity, supports row-filter/data-masking, etc.


Type of resources vary across services/applications, as seen in the table below:

Service

Resources

Apache Hive

databases, tables, columns, udfs

Apache Kafka

topics

Apache Solr

collections

AWS S3

buckets, objects

ADLS-Gen2

storage-accounts, containers, objects

Azure PowerBI

workspaces

Google BigQuery

projects, datasets, tables, columns

Snowflake

databases, schemas, tables, columns, warehouses

Trino

catalogs, schemas, tables, columns, procedures

...


Permissions

A permission is another fundamental element in the Apache Ranger policy model. A permission is an action performed on a resource, like reading a file, creating a directory, querying a table, or publishing a message to a topic. Apache Ranger policy model captures details of permissions of a service in a declarative way – details like which permissions are applicable to specific resource types, implied permissions, etc.


Like resources, list of permissions varies across services/applications, as seen in the table below:

Service

Permissions

Apache Hive

create, alter, drop, select, insert, ..

Apache Kafka

publish, consume, create, delete, describe, configure, ..

Apache Solr

query, update, others, Solr admin

AWS S3

read, write, delete, ..

ADLS-Gen2

read, write, delete, ..

Azure PowerBI

contributor, member, admin, none

Google BigQuery

project-list, dataset-create, table-create, table-list, query, ..

Snowflake

CreateSchema, CreateTable, Select, Insert, Update, ..

Trino

create, alter, drop, select, insert, ..

...


Users, Groups, Roles

Apache Ranger enables authorization policies to be set up to allow/deny permissions to users, groups, and roles. Users and groups are typically obtained from an enterprise directory like AD/LDAP. Apache Ranger user-sync module handles details of bringing users and groups from sources like LDAP/AD/OS, and keeping up with the changes in the sources - like addition of users and groups, addition/removal of a user from a group.


Apache Ranger user-sync supports retrieving attributes of users and groups as well. Such attributes, like dept/location/site-id, can be used in authorization policies to allow/deny access to resources, and set up row-filters that restrict users to access relevant subset of data. More on this later in this document.


In addition to users and groups, Apache Ranger supports roles to be used in authorization policies. A role in Apache Ranger is a grouping of users, groups, and other roles. Roles can be managed using Apache Ranger UI and REST APIs by authorized users. Role based authorization is widely used in enterprises and having support for roles in Apache Ranger makes it possible to use well established enterprise security practices in Apache Ranger authorization policies.


Delegated Admin

Apache Ranger enables decentralization of authorization policies management with support for delegated-admin feature. A set of users, groups and roles can be granted permission, via an Apache Ranger policy (what else!), to manage authorization policies for a subset of resources and permissions. For example, users in finance-admin group can be granted permissions to manage authorization policies for contents of Snowflake database named finance, and AWS S3 objects under s3://mybucket/dept/finance. This offers a scalable approach to manage authorization in large deployments.


Security Zone

Apache Ranger supports security zones to enable multi-tenancy within an organization where admins from different lines of businesses can manage security policies for their own resources. For example, data that belongs to the sales team can be managed by administrators of the sales team, similarly data of marketing, sales, operations teams can be managed by respective administrators.


Also, security zones can be used to isolate resources based on purpose. For example, it is common for a data lake to have distinct areas and authorization policies for test data, unprocessed/raw data, semi-processed data, and production data. Apache Ranger makes it easier to manage security policies in such deployments with use of security zones like:

  • Test zone

  • Landing zone

  • Staging zone

  • Production zone


A security zone can contain resources from multiple services/applications, like AWS S3, ADLS-Gen2, GCS, Snowflake, Amazon Redshift, Postgres, Apache Hadoop, Apache Hive, Apache HBase, Apache Kafka. This makes it easier to set up consistent authorization policies across multiple services by a set of administrators designated for each security zone. 


Allow, Deny, Exceptions

In addition to authorization policies that can grant access to resources, Apache Ranger also enables policies to be setup to:

  • deny access to users/groups/roles on resources 

  • exclude a subset of users from accesses allowed/denied above

  • deny all access to specific resources other than the ones allowed in the policy


This makes it easier to set up policies to protect sensitive resources.


Wildcards, macros, variables in resource names

Apache Ranger policies support use of wildcards, macros, and variables in resource names. This makes it possible to use small number of policies for a large number of resources, as shown below:


test_*

matches all resources having name that start with test_

/home/{USER}

a path under /home having name of current user

/dept/$USER.dept}}

a path under /dept having name of current user’s department


Policy validity schedule

Apache Ranger enables policies to be effective only for specific time schedules. This feature can be used to create policies that need to be effective at a future time, for example to allow access to revenue reports for a wider audience only after a specific time. This feature can also be used to allow temporary access to specific users/groups/roles, with a specific start and end times.

Attribute-based access control

Apache Ranger enables use of user, group, resource, classification, and the environment attributes in authorization policies.


ABAC makes it possible to express authorization policies without prior knowledge of specific resources, specific users – which helps avoid the need for new policies as new resources or users are introduced.  For example:

  • allow each user to access all tables owned by them, using{OWNER} macro in the following policy:

resource

database=*, table=*

users

{OWNER}

permissions

all


  • allow users to access their department data in AWS S3, by using user attribute $USER.dept}} in the following policy:

resource

bucket=mycompany, object=/data/$USER.dept}}/*

users

{USER}

permissions

read, write


  • allow users in mktg group to access PII data of email type, by using tag attribute TAG.piiType in the following policy:

resource

tag=PII

groups

mktg

condition

TAG.piiType == 'email'

permissions

select


  • tables with SENSITIVE classification should be accessible only by users having privileges for that sensitive level

resource

tag=SENSITIVE

groups

public

condition

TAG.sensitiveLevel <= USER.allowedSensitiveLevel

permissions

select 


ABAC can be used to set up row-filters as well. For example:

  • restrict managers to access only records of employees in their department, by using user attribute $USER.deptCode}} in the following row-filter policy:

resource

database=hr, table=employee

groups

managers

row-filter

dept_code = $USER.deptCode}}


  • restrict store administrators to access only data of the stores they manage, by using user attribute $USER.empId}} and look up table store_admins in  following row-filter policy:

resource

database=sales, table=store_sales

groups

store-admin

row-filter

store_id in (select sa.store_id

               from store_admins sa

              where sa.admin_id = $USER.empId}})


Resource-based access control

Apache Ranger enables setting up policies to grant or deny permissions to users/group/roles based on specific resource names, like:

Service

Resource

Permissions

Apache Hive

database

sales

table

order_data

column

order_amount


select

Apache Kafka

topic

finance


publish, consume

AWS S3

bucket

mycompany

path

/home/{USER}/


read, write, delete

ADLS-Gen2

storage-account

mycompany

container

home

path

/{USER}


read, write, delete

...


Tag-based access control

In addition to authorization policies on resources, Apache Ranger enables policies to be set up on classifications (tags) associated with resources. This feature enables enterprises to separate responsibility of classification of resources (PII, PCI, PHI, credit card number, etc.) from setting up access-control policies. Classifications created, by a team of data stewards and tools that scan data for sensitive information, can be leveraged to drive authorization to access the resources.


Authorization policies on the classifications themselves, instead of directly on the resources, will ensure that appropriate policies will automatically be applied as classifications are added , removed, and updated on resources. Also, a single tag-based policy (for example on PII) can be used to authorize access to resources across multiple services like AWS S3, ADLS-Gen2, Snowflake, Databricks SQL, Apache Hive, Apache HBase, Apache Kafka. This can significantly reduce the complexity in managing authorization policies.

Data masking

Apache Ranger data-masking policies enable enterprises to allow access to sensitive data suitably masked depending on the context in which a user accesses the data. Some users will need the data without masking, while some other users can only be allowed to see partial or masked or transformed value. While authorization policies can be used to either allow or deny access to certain data, data-masking policies enable dynamically mask sensitive data as users access the data, for example to ensure that:

  • analysts have access to only specific part of birthday (year or month or day)

  • only last 4 digits of a national id are available to customer service representatives

  • only salary ranges of employees (i.e., not the salary) are available to analysts


In addition to supporting data-masking policies on resources, like columns in Apache Hive/Snowflake/Databricks SQL/Presto, Apache Ranger enables setting up data-masking policies based on classifications (tags) associated with resources. This can significantly reduce the complexity in managing masking policies. In addition, tag-based masking policies leverage classifications added to resources by data stewards and tools that scan data for sensitive information.

Row-filter

Apache Ranger row-filter policies enable enterprises to allow users to access only a subset of data depending upon the context in which a user accesses the data. When a table having a row-filter is accessed by the user, only a subset of rows will be visible to the user – depending upon the filter setup in row-filter policy. Row-filters can be used for example to ensure that:

  • data of customers residing in a country is available only to analysts authorized to access the country’s data

  • a store manager has access to only data relevant to the store she/he works in

  • analysts don’t have access to sensitive records

Access audit logs

Apache Ranger generates audit logs of accesses to resources protected by Apache Ranger authorization. Apache Ranger can be configured to store audit logs in multiple destinations, including Solr, HDFS, AWS S3, AWS CloudWatch, ADLS-Gen2, Elasticsearch. Audit logs generated by Apache Ranger include following details, which can help enterprises to satisfy various compliance requirements:

  • resource accessed; action performed; was access allowed

  • time of access, tags associated with the resource (PII, PCI, PHI, ..)

  • who performed the access, IP address from which the access was performed

  • ID of Apache Ranger policy that allowed or denied the access


Apache Ranger provides an interactive user interface to view audit logs stored in Solr, Elasticsearch or AWS CloudWatch, with search capabilities to look for access audits for specific resources, specific users, client IP addresses, within a given time frame, specific classifications.


Apache Ranger audit logs can be stored in ORC or JSON formats, which can then be loaded into various tools for analysis.

References