Your ServiceNow instance integrates with an external HR system via OAuth 2.0 for employee data sync.

What the Interviewer Wants to Hear

Intermittent 401 errors during peak hours suggest timing-related issues like token expiry mid-sync, clock skew between systems, or concurrent refresh races. Diagnosis starts with logs: filter Integration Logs and System Logs for 'OAuth' or '401' with correlation IDs. Check token expiry via REST API Explorer as integration user. Verify instance time sync (gs.getSession().getTimeZone()). Test load with Scripted REST API simulating 500 calls. Review OAuth provider config for grant type (client credentials?) and scope. Impersonate user to rule out ACLs/roles. External system audit logs confirm no blocks.

The Solution

Problematic Code (typical race-condition prone):

var OAuthUtil = Class.create();
OAuthUtil.prototype = {
    initialize: function() {},
    getToken: function() {
        var r = new sn_ws.RESTMessageV2();
        r.setEndpoint('https://hr.example.com/oauth/token');
        r.setHttpMethod('POST');
        r.setRequestHeader('Authorization', 'Basic ' + gs.getProperty('hr.client.secret'));
        r.setRequestBody('grant_type=client_credentials');
        var response = r.execute();
        if (response.getStatusCode() == 200) {
            var token = JSON.parse(response.getBody()).access_token;
            gs.setProperty('hr.oauth.token', token); // Unsafe shared prop
            return token;
        }
        gs.error('Token fetch failed');
    },
    callHRAPI: function() {
        var token = gs.getProperty('hr.oauth.token');
        // Use token - fails if expired mid-batch
    },
    type: 'OAuthUtil'
};

Fixed Code (thread-safe with expiry check):

var SecureOAuthUtil = Class.create();
SecureOAuthUtil.prototype = {
    initialize: function() {
        this.tokenTable = 'x_custom_oauth_tokens'; // Illustrative only — prefer the
        // platform OAuth 2.0 credential store (OAuth Entity / GlideOAuthClient),
        // which handles refresh + encrypted storage. If you must use a table,
        // make the token field Password2 (encrypted), never plain text.
        this.expiryBuffer = 300; // 5min buffer
    },
    getToken: function() {
        var gr = new GlideRecord(this.tokenTable);
        gr.addQuery('name', 'hr_api');
        gr.query();
        if (gr.next()) {
            var expires = new GlideDateTime(gr.getValue('expires_at'));
            var cutoff = new GlideDateTime(); // now
            cutoff.addSeconds(this.expiryBuffer);
            if (expires.after(cutoff)) { // compare with after()/before(),
                                         // never with > on GlideDateTime
                return gr.getValue('access_token');
            }
        }
        return this.refreshToken();
    },
    refreshToken: function() {
        var r = new sn_ws.RESTMessageV2();
        // ... (token request as above)
        var tokenData = JSON.parse(r.getBody());
        var gr = new GlideRecord(this.tokenTable);
        gr.get('name', 'hr_api') || gr.initialize();
        gr.access_token = tokenData.access_token;
        gr.expires_at = new GlideDateTime();
        gr.expires_at.addSeconds(tokenData.expires_in - this.expiryBuffer);
        gr.update();
        return tokenData.access_token;
    },
    type: 'SecureOAuthUtil'
};

Senior Considerations

• Diagnosis Depth: Reading the logs is the baseline; go further by correlating ECC Queue entries, Flow Contexts, and external-system timestamps, and by scripting a load test.
• Options & Trade-offs: 1) Platform OAuth credential store / IntegrationHub credential (preferred): the instance manages refresh and encrypted storage. 2) Custom token table (above): workable for exotic providers, but you own expiry, races, and encryption. 3) Mutex Script Include: prevents refresh races; added complexity for high TPS.
• Production Hardening: Event Registry alerts on 401>5%, Performance Analytics on sync latency, least-privilege role (itil + custom API), rotate secrets via Vault, A/B test in subprod.
• Edge Cases: Clock skew (use NTP), provider revokes, network partitions (retry w/ exp backoff), token blacklist.
• Why This Matters: 15% failure cascades to business (onboarding delays); scales to 10k/hr without rework.