chore(graph): disable HTTP or eventhandlers by configuration

[pbleser-oc]

Description

In the scope of the broader issue #1312, this PR deals with performing those changes for the graph service, namely to add the ability to disable the HTTP API or to disable the events API handler by configuration.

It also adds numerous metrics, and tests for the events processing.

Making Event Processing and HTTP Optional (#1312)

The previous implementation was combining the HTTP server service and the events consumption, which is why this PR refactors the composition of those services:

• the event consumption has been moved into its own service
• the identity.Backend is created beforehand, and then injected as a collaborator in both the HTTP service as well as the event consumer service

To encourage re-use in latter implementations and changes, it also introduces two top-level package changes:

• internal/eventstest/events_test_helpers: contains a TestBus implementation to unit-test event consumers without the need to engage NATS
• internal/metricstest/metrics_test_helpers: contains assertion functions to test Prometheus metrics

Run-down of changes for this aspect:

• add the ability to disable the HTTP API handler (GRAPH_HTTP_DISABLED)
• add the ability to disable the Events API handler (GRAPH_EVENTS_DISABLE_CONSUMER)
• add metrics, especially for event processing
• add metrics to the documentation
• introduces a new top-level packages internal/eventstest and internal/metricstest with utilities for testing event consumers and Prometheus metrics, respectively

Adding Metrics

Introducing gowrap as a build-time tool to generate interface delegate structs from templates:

• added as a make go-generate target in services/graph,
• added as a build-time dependency in .bingo/

Introduces LDAP client abstraction interface to be able to wrap the go-ldap client API with metrics transparently (and possibly hooks and such in the future), in order to use delegation patterns to measure the time LDAP (client) operations take to finish, as well as to track their results (success, failure, not-found).

Has two implementations that are generated using gowrap:

• a go-ldap adapter implementation that directly delegates
• a time measuring and metrics collecting implementation that delegates to another LdapClient

The metrics collecting one is disabled by default, can be enabled with GRAPH_LDAP_METRICS_DISABLE=false

It collects durations of outbound LDAP client operations into a histogram, as well as the number of concurrent outbound LDAP operations in a gauge.

Add an HTTP middleware that measures how long Graph HTTP API requests take, storing taken time into a histogram along with labels for

• method,
• path pattern (from the chi routes),
• Graph API version prefix,
• Graph API resource name,
• and the resulting status code.

It also tracks the number of concurrent inbound Graph API HTTP requests using a gauge.

Disabled by default, can be enabled with GRAPH_HTTP_METRICS_DISABLE=false

Add Backend and EducationBackend delegate implementations that measure execution time on the level of the higher API call operations there (CreateUser, DeleteUser, ..., CreateSchool, ...), generated using gowrap.

Disabled by default, can be enabled with GRAPH_IDENTITY_BACKEND_METRICS_DISABLE=false

Made some internal changes to how some of the LDAP client API operations work in the LDAP backend:

• check whether searches for a singular entry returns more than one result, in which case a new error TooManyResults is returned, instead of leaving that undetected, blindly taking the first result, and potentially risking data inconsistencies
• DeleteUser() does not return an error any more when the user to delete cannot be found in LDAP: instead, it now also returns a set of bools following the 'ok' idiom, and callers can deal with whether that is supposed to end up in an error or not on their level
• GetUser() and UpdateUser() also go not return an error when the user entry is not found in LDAP; instead, they returns nil for the user, which must now be checked for nilness by callers, and dealt with as they see fit depending on the context of the operation
• introduced custom types for booleans: Supported and Found, in order to make the semantics of just returning bools less confusing; unfortunately, they had to be moved to a package of its own to avoid package import cycles

Callers have been modified accordingly, checking for nilness of the returned user in order to behave the same way as before, returning an ItemNotFound error on their (higher) level.

Improve the loggers in identity backends by adding attributes for their request targets (Reva gateway address or LDAP URI, respectively).

Also add a "backend" attribute for all Graph API logs (set to "ldap" or "cs3"), to help debug potential issues.

The LDAP identity backend logger also has two new attributes to help debugging with logs:

• write (bool): whether write operations are enabled
• refint (bool): whether refint is enabled or not

Also adds a dedicated counter for user password change operations.

Additional boy-scouting

• in identity/backend.go: modify the UpdateLastSignInDate function to return a bool in addition to the error to clarify whether the operation was even attempted or not, to be able to detect when the operation is unsupported, and log errors (or not) accordingly
• add a constructor func for the CS3 backend
• add a constructor func for the LDAP backend
• in the LDAP identity backend, in searchLDAPEntryByFilter (used by all search/get public functions), errors that occur when performing LDAP SEARCH operations were blindly mapped to a ItemNotFound error, instead of being analyzed as it could be caused by a technical error
• in the requireadmin middleware, add debug logging to explain why a request is denied
• when an LDAP password change fails because the user entry was not found in LDAP, we now have a log message that tracks that

Related Issue

• fixes a part of [FR] Starting Event Listeners should be optional #1312

Motivation and Context

Details can be found in #1312

How Has This Been Tested?

Tested the HTTP API (when enabled) using:

xh --verify=no -a alan:demo --https :9200/graph/v1.0/me

and

xh --verify=no -a alan:demo --https :9200/graph/v1.0/users/b1f74ec4-dd7e-11ef-a543-03775734d0f7

and

xh --verify=no -a admin:admin --https :9200/graph/v1.0/users \$orderby=='displayName asc'

and

xh --verify=no -a admin:admin --https POST :9200/graph/v1.0/users accountEnabled:=true displayName="Jane Doe" onPremisesSamAccountName=jdoe mail="jdoe@example.com" passwordProfile:='{"password": "secret12345"}'

Tested the event API (when enabled) using:

echo -n '{"metadata":{"eventtype":"events.UserSignedIn"},"payload":"'$(echo -n '{"Executant":{"opaque_id":"alan"}}'|base64)'"}'|nats pub main-queue

Tested combinations of enabling/disabling the events and HTTP APIs.
When both are disabled, the service refuses to start and exits early with an error message.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Technical debt
• Tests only (no source changes)

Checklist:

• Code changes
• Unit tests added
• Acceptance tests added
• Documentation added

[codacy-production]

Not up to standards ⛔

🔴 Issues 2 critical

Alerts:
⚠ 2 issues (≤ 0 issues of at least minor severity)

Results:
2 new issues

Category	Results
Security	2 critical

View in Codacy

🟢 Metrics 385 complexity · 284 duplication

Metric	Results
Complexity	385
Duplication	284

View in Codacy

_{NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.}