Relevant versions: This information pertains to Artifactory version 7.x
Written in Go, Metadata is a microservice that comes bundled with Artifactory 7.x. It's used to store metadata information packages from JFrog services, such as Artifactory and Xray, and has its own configuration settings and logs files. As Artifactory is written in a different language (Java), this can create several problems. Accordingly, when any issue arises with Metadata, start your troubleshooting by having a look in the $ARTIFACTORY_HOME/var/log directory at the following files:
If your Artifactory instance is running with an external Oracle DB, you might encounter this error:
1. [database]
This means that there's a missing Libaio library, which is required to work with the Oracle DB. To resolve this issue, you'll need to install it by following the instructions available HERE.
2. An error occurred when converting a Java Database Connectivity (JDBC) connection string to a GO-style connection string. For example:
Here the connection string was as follows:
The workaround is to go inside your system.yaml and, in the metadata.database section, add a Go connection string with go: prefix, and then add a go-style connection string instead of JDBC connection string. The details on how to build a connection string for Oracle are available HERE. For example, if the connection string is oracle://user:passw@service_name, you'll need to set the value to go:oracle://user:passw@service_name. ֿ
3. Here's a different type of error:
This indicates a missing SSL/TLS certificate. To fix this, on your Artifactory machine, you'll need to add the appropriate certificate file in your /etc/ssl/certs folder.
4. Another common error is when the database password inside of the Go connection string's metadata.database.url is not encrypted after startup. The error above occurs because it's impossible to encrypt only a part of the connection string safely, as there's no standard way of writing those in Go, as there is in the JDBC. To fix this, we've created a placeholder replacement for this configuration key only. Here's how to use it:
When reading the Go connection string, ${shared.database.password} will be read as very_secret_password (the file contents won't change) and very_secret_password will be encrypted in the physical file by the router service as usual. You may use any configuration key in the placeholder, even environment variables.
5. Master Data Services (MDS) fails with the following error:
This occurs because the password to connect to the PostgreSQL database contained the backslash character, which failed to parse. There are a few possible workarounds:
For example:
This is another version of the JDBC which doesn't contain the backslash and will be accepted by the Metadata service.
Published: Dec. 28, 2020
Last updated: May. 31, 2020
Keywords: metadata, database
Written in Go, Metadata is a microservice that comes bundled with Artifactory 7.x. It's used to store metadata information packages from JFrog services, such as Artifactory and Xray, and has its own configuration settings and logs files. As Artifactory is written in a different language (Java), this can create several problems. Accordingly, when any issue arises with Metadata, start your troubleshooting by having a look in the $ARTIFACTORY_HOME/var/log directory at the following files:
- metadata-service.log
- metadata-request.log
- console.log
If your Artifactory instance is running with an external Oracle DB, you might encounter this error:
2020-03-19T18:26:17.366Z [jfmd ] [PANIC] [4302998e04479677] [database_bearer.go:51 ] [main ] - Could not initialize database (db config: {oracle ARTIFACTORY_TEST/***@(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=pepldr00237.corp.pep.pvt)(PORT=60003))(CONNECT_DATA=(SERVER=DEDICATED)(SID=stm1d)))}): ORA-00000: DPI-1047: Cannot locate a 64-bit Oracle Client library: "The specified module could not be found". See https://oracle.github.io/odpi/doc/installation.html#windows for help
error connecting to database. giving up
jfrog.com/metadata/services/common.(*databaseBearer).init
metadata/services/common/database_bearer.go:84
jfrog.com/metadata/services/common.NewDatabaseBearer
metadata/services/common/database_bearer.go:49
…
/database_bearer.go:51 +0x2a8
main.main()
metadata/metadata.go:29 +0x435
1. [database]
This means that there's a missing Libaio library, which is required to work with the Oracle DB. To resolve this issue, you'll need to install it by following the instructions available HERE.
2. An error occurred when converting a Java Database Connectivity (JDBC) connection string to a GO-style connection string. For example:
error reading configuration from system.yaml : reading db properties failed: could not derive db properties from system yaml: Type: , Url: : Could not set db properties to application configuration: Could not parse jdbc url's connect part: ldap://uoid.rsv.se:7389/ ZV3T001,cn=OracleContext,dc=rsv,dc=se goroutine 1 [running]:
Here the connection string was as follows:
jdbc:oracle:thin:@ldap://mydomain.nex.co:7389/ ZV3T001,cn=OracleContext,dc=rsv,dc=seMetadata tried but failed to convert the JDBC connection string to a Go connection string.
The workaround is to go inside your system.yaml and, in the metadata.database section, add a Go connection string with go: prefix, and then add a go-style connection string instead of JDBC connection string. The details on how to build a connection string for Oracle are available HERE. For example, if the connection string is oracle://user:passw@service_name, you'll need to set the value to go:oracle://user:passw@service_name. ֿ
3. Here's a different type of error:
2020-03-22T18:25:10.663Z [jfmd ] [ERROR] [4104ec4b9d67964e] [database_bearer.go:51 ] [main ] - Could not initialize database (db config: {mysql artifactory:***@tcp(adim-mysql.cvooa7fn0wpy.us-west-2.rds.amazonaws.com:3306)/artdb?charset=utf8&tls=true&parseTime=true&clientFoundRows=true}): x509: certificate signed by unknown authority error connecting to database. giving up
jfrog.com/metadata/services/common.(*databaseBearer).init
/src/jfrog.com/metadata/services/common/database_bearer.go:84
jfrog.com/metadata/services/common.NewDatabaseBearer
/src/jfrog.com/metadata/services/common/database_bearer.go:49
main.main
/src/jfrog.com/metadata/metadata.go:29
runtime.main
/src/runtime/proc.go:203
runtime.goexit
/src/runtime/asm_amd64.s:1357 This indicates a missing SSL/TLS certificate. To fix this, on your Artifactory machine, you'll need to add the appropriate certificate file in your /etc/ssl/certs folder.
4. Another common error is when the database password inside of the Go connection string's metadata.database.url is not encrypted after startup. The error above occurs because it's impossible to encrypt only a part of the connection string safely, as there's no standard way of writing those in Go, as there is in the JDBC. To fix this, we've created a placeholder replacement for this configuration key only. Here's how to use it:
shared:
database:
driver: org.postgresql.Driver
password: very_secret_password
type: postgresql
url: "jdbc:postgresql://mydatabase.net:5432/artifactory"
username: artifactory
metadata:
database:
url: go:user='artifactory' password='${shared.database.password}' dbname=artifactory host=mydatabase.net port=5432 sslmode=disable When reading the Go connection string, ${shared.database.password} will be read as very_secret_password (the file contents won't change) and very_secret_password will be encrypted in the physical file by the router service as usual. You may use any configuration key in the placeholder, even environment variables.
5. Master Data Services (MDS) fails with the following error:
2020-06-15T07:25:47.196Z [jfmd ] [INFO ] [60ebe5752c965a8d] [database_bearer.go:84 ] [main ] - Connecting to (db config: {postgresql user='artifactory' password='***' dbname=artifactory host=127.0.0.1 port=5432 sslmode=disable}) [database]
2020-06-15T07:25:47.214Z [jfmd ] [PANIC] [60ebe5752c965a8d] [database_bearer.go:68 ] [main ] - Could not initialize database (db config: {postgresql user='artifactory' password='***' dbname=artifactory host=127.0.0.1 port=5432 sslmode=disable}): error connecting to database
jfrog.com/metadata/services/common/db.(*databaseBearer).init
/src/jfrog.com/metadata/services/common/db/database_bearer.go:114
jfrog.com/metadata/services/common/db.NewDatabaseBearer
/src/jfrog.com/metadata/services/common/db/database_bearer.go:66
main.main/src/jfrog.com/metadata/metadata.go:38
runtime.main
/src/runtime/proc.go:203
runtime.goexit
/src/runtime/asm_amd64.s:1373
goroutine 1 [running]:
runtime/debug.Stack(0x38, 0xc000160300, 0xc00032c200)
/src/runtime/debug/stack.go:24 +0x9d
jfrog.com/jfrog-go-commons/pkg/log.(*standardLogger).Panicfc(0xc000554ac0, 0x16058a0, 0xc0004d9bf0, 0x1390d88, 0x32, 0xc00032c200, 0x2, 0x2)
/src/jfrog.com/go-commons/pkg/log/standard_logger.go:42 +0x6a
jfrog.com/metadata/services/common/db.NewDatabaseBearer(0x16058a0, 0xc0004d9bf0, 0x16064e0, 0xc000090b90, 0x160a760, 0xc0004e5810, 0x15fd860, 0xc0000bf2e8, 0x15e0aa0, 0xc000554a90, ...)
/src/jfrog.com/metadata/services/common/db/database_bearer.go:68 +0x2d4
main.main()
/src/jfrog.com/metadata/metadata.go:38 +0x5b7
[database] This occurs because the password to connect to the PostgreSQL database contained the backslash character, which failed to parse. There are a few possible workarounds:
- Create a new password without removing the backslash character from the password (Note: This change must also be made on the DB side.)
- Provide the password with an encoded character (instead of the '').
- Similar to the workaround for issue #2 above, provide the Go connection string explicitly to the Metadata service, which is inside your YAML configuration file.
6. If you're the MSSQL/SQL server DB as your external DB, it's very common that the connection string will contain a backslash in the URL between the server name and the instance name. For example: JFrogsqlserver, which can also create an issue for the Metadata server. To resolve this, here are two options:
- Modify your JDBC URL as follows:
url:jdbc:sqlserver://<your-db- dns>:<port>;instanceName=<instan-name>;sendStringParametersAsUnicode=false;databaseName=<database-name>
For example:
url:jdbc:sqlserver://vtcFxzh@myartifactorydb:1385;instanceName=ARTIFACTORY;sendStringParametersAsUnicode=false;databaseName=artifactory
This is another version of the JDBC which doesn't contain the backslash and will be accepted by the Metadata service.
- Another option is to provide an MSSQL Go connection string (as specified in the examples here) separately to the Metadata service.
Published: Dec. 28, 2020
Last updated: May. 31, 2020
Keywords: metadata, database