Introduction
This article outlines how to troubleshoot and resolve a Mission Control (MC) 403 Forbidden Error that is accompanied by a JSON parsing failure. This pattern indicates that MC's internal traffic to a target JFrog Platform Deployment (JPD) is being erroneously routed through an external Forward Proxy Server and subsequently blocked.
The fix involves identifying that the internal request never reaches its target and correcting the platform's proxy configuration to bypass the external proxy for Mission Control (jfmc) communication.
1. Initial Troubleshooting: Why DEBUG Logging is Essential
When the mc−request−out.log shows a 403|Forbidden error, the immediate suspect is often a permission issue at the target JPD. However, to quickly differentiate a target permission error from a proxy interception, follow this logic:
-
Check Target Logs: If the request was refused by the Target JPD, the target's request logs (e.g., artifactory-request.log) would show the incoming request and the 403 status. If the target logs show nothing, the request was blocked upstream, likely by a proxy.
-
Enable DEBUG Logging: Only the DEBUG level captures the complete response body, revealing the critical clue that points away from the target JPD:
-
JSON Parsing Error: The response body contains the non-JSON character < (code 60), indicating an HTML error page (not a standard JPD JSON error).
-
Proxy Selection: The logs explicitly show the MC component selected and used an external proxy.
-
Therefore, enabling DEBUG logging on the source node where Mission Control (mc) is running is mandatory to identify the root cause.
Modify the Logback Configuration
On the affected source node:
-
Locate the Mission Control logback configuration file:$JFROG_HOME/artifactory/var/etc/mc/logback.xml
-
Change the logging level for the org.jfrog.mc logger.-
AS-IS:<logger name="org.jfrog.mc" level="INFO"/>
-
TO-BE:<logger name="org.jfrog.mc" level="DEBUG"/>
Note: Changing the log level to DEBUG does not require a service restart.
-
2. Collect Logs
-
Allow the error to reproduce.
-
Gather all mc−xxxxx.log files from that node for further analysis.
3. Diagnosis: External Proxy Misdirection
The DEBUG logs confirm the internal request was misdirected through an external proxy.
Example Log Entries:
2025-10-01T20:14:09.278Z [jfmc ] [DEBUG] [ ] [c.u.RepositoryProxySelector:31] [nitor-remoteTopology] - Proxy 'Proxy(host=region1-admin-proxy-vip.xyz.com, port=3128, username=null)' found.
...
2025-10-01T20:18:39.757Z|...|http://region2-artifactory-vip.xyz.com:8082/access/api/v1/topology?enriched=true|GET|403|Forbidden|1
2025-10-01T20:18:39.757Z [jfmc ] [DEBUG] [...] - Received error reasonPhrase with non-json content: 'Forbidden'com.fasterxml.jackson.core.JsonParseException: Unexpected character ('<' (code 60)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
Conclusion: The DEBUG log shows an external proxy (region1-admin-proxy-vip.xyz.com) was selected, and the JSON parse failure confirms the proxy is blocking the request with an HTML error page.
4. Resolution: Remove jfmc from Proxy Services List and Update Configuration
The issue is that the proxy configuration is incorrectly applied to the Mission Control (jfmc) service. MC's communication must be treated as internal and must bypass the forward proxy.
Action: Update Proxy Configuration
Modify your JFrog Platform's proxy configuration to remove jfmc from the list of services associated with your proxy.
|
Status |
Services Tag |
Effect |
|
❌ Incorrect (Current State) |
<services>jfrt,jfmc</services> |
Forces jfmc internal traffic through the external proxy, causing the 403 error. |
|
✅ Correct (Target State) |
<services>jfrt</services> |
Only applies the proxy to Artifactory (jfrt) for external access, allowing jfmc internal traffic to proceed directly. |
Implementation Methods
This change can be applied using one of the following methods:
Method A: JFrog Platform User Interface (UI)
-
Navigate to Administration (Gear Icon) -> Configuration (or General) -> Proxies.
-
Edit the relevant proxy configuration (e.g., proxy1).
-
Locate the Services or Apply to Services section.
-
Deselect or remove the Mission Control (jfmc) service from the list.
-
Save the configuration.
For further reference, consult the official documentation: Define Proxy Server.
Method B: Configuration Descriptor (XML/YAML API)
-
Retrieve the current system configuration descriptor using the Artifactory REST API.
-
Modify the file by removing jfmc from the services list within the proxy definition.
-
Apply the modified configuration back to the Platform using the appropriate Artifactory REST API call.
Once the configuration is corrected, Mission Control traffic will be direct and internal, resolving the 403 error.
5. Final Step: Revert Logging to INFO
Once the issue is resolved, revert the logging level to INFO to prevent unnecessary disk usage.
-
On the affected node, open $JFROG_HOME/artifactory/var/etc/mc/logback.xml. -
Change the logging level back:-
TO-BE (Revert):<logger name="org.jfrog.mc" level="INFO"/>
-