‹ All interview questions

Architecture

Your enterprise customer has 5,000 users across 3 business units (HR, Finance, IT) on a single ServiceNow…

Senior · ServiceNow · Jan 2026

The question

Your enterprise customer has 5,000 users across 3 business units (HR, Finance, IT) on a single ServiceNow instance. Recently, HR workflows are spiking CPU usage to 90% during payroll runs (affecting 2,000 users), causing IT incidents to timeout globally. Logs show cross-domain queries hitting 10M rows/day without isolation. Customer demands data isolation for compliance (GDPR/SOX) but wants to retain shared configs. You're seeing 20% monthly growth in incidents. Question: Diagnose the root cause systematically. Propose 3 multi-instance strategies with trade-offs. Provide a script showing the performance bottleneck and your fix. Explain production rollout considerations for zero-downtime.

What the Interviewer Wants to Hear

Root cause diagnosis starts with triage: Check System Performance > Transactions for slow queries, then System Logs > Errors for domain leaks. Use System Diagnostics > Slow Queries to identify unfiltered cross-domain scans (e.g., incident.getRowCount() without domain.addQuery()). Metrics reveal HR payroll workflows (e.g., Business Rule on payroll table) querying global incident tables, bypassing domain separation—common in single-instance setups lacking row-level security (RLS).

Key issues: No domain isolation (shared sys_id prefixes leak), resource contention (CPU throttling), scalability limits (single DB bottleneck at 10M queries/day).

The Solution

Diagnosis Script (Problem Demo): This GlideAggregate reveals unfiltered cross-domain query bloat.

var ga = new GlideAggregate('incident');
ga.addQuery('active', true);
ga.addAggregate('COUNT');
ga.groupBy('sys_domain'); // Group to count per domain
// PROBLEM: No domain filter - scans ALL domains (10M rows)
ga.query();
var count = 0, domains = 0;
while (ga.next()) {
  count += ga.getAggregate('COUNT');
  domains++; // Each group = one distinct domain
}
gs.info('Total Incidents: ' + count + ', Domains Hit: ' + domains);

Solution Script (Multi-Instance Domain Isolation): Fix adds tenant-aware filtering + async partitioning.

var ga = new GlideAggregate('incident');
ga.addQuery('active', true);
ga.addQuery('sys_domain', gs.getUser().getDomainID()); // FIX: enforce domain filtering
ga.addAggregate('COUNT');
ga.query();
var count = ga.next() ? ga.getAggregate('COUNT') : 0;
gs.info('Filtered Incidents: ' + count);

3 Strategy Options with Trade-offs

  • Option 1: Enhance Domain Separation (Single Instance): Activate full domain sep (data/process/UI). Pros: No migration, shared configs. Cons: Logical isolation only (query leaks possible), scales to ~50K users. Cost: Low. Use for 80% cases.
  • Option 2: Separate Instances per Business Unit (Recommended): dedicated instances (HR/Finance/IT) integrated via Integration Hub. Pros: true isolation, independent sizing and upgrade schedules. Cons: config sync overhead, additional licensing. Ideal for compliance + growth.
  • Option 3: Per-Tenant Dedicated Instances: Full separation. Pros: Max isolation/security. Cons: High cost (5x infra), no shared metadata, complex failover. Avoid unless regulated industry mandates.

Senior Considerations

  • Why Separate Instances Win Here: Balances hard isolation with configurability (shared apps promoted from a common repo). Trade-off: cross-instance reporting and lookups add latency—mitigate with event-driven integrations via Integration Hub rather than synchronous calls.
  • Production Rollout (Zero-Downtime): Stage the cutover: clone prod to the new instances, migrate configuration via Update Sets/app repo and data via Transform Maps (scripted, batched), and load test with production-like concurrency before go-live. Monitor via PA dashboards (CPU <70%). Fallback: keep the original instance warm until sign-off.
  • Edge Cases/Failure Modes: Domain path explosion (use sys_domain_path indexing). Burst spikes: work with ServiceNow on instance sizing — customers cannot add or auto-scale nodes themselves. Audit: Enable Field Watcher for leaks. Great answers quantify the improvement from their own sub-prod measurements and discuss cost trade-offs honestly.
  • Great vs. Good: A good answer lists the options; a great one measures actual impact in a sub-prod test, covers rollback, and cites the official domain separation documentation. Expect the probe: 'How would you test isolation pre-prod?'