JFrog and Qwak
JFrog announced a new integration with Qwak, a fully managed ML Platform, that brings machine learning models alongside traditional software development processes to streamline, accelerate, and scale the secure delivery of ML applications. Read the press release.
For a deeper look at the integration between the JFrog Platform and Qwak and how it works, read this blog.
How to get started
Connecting JFrog and Qwak
New to JFrog?
Start free with JFrog here.
New to Qwak?
Start free with Qwak here.
Once you’re connected to both platforms or you are a current user of both platforms, you are ready to experience the benefits of JFrog + Qwak.
Connection Steps
An Admin token is required for establishing a connection between JFrog and Qwak.
To integrate JFrog with Qwak, you’ll need:
- JFrog Base URL: This is the web address of your JFrog instance, such as https://qwak.jfrog.io.
- Access Token: An access token with Administrator rights is required. Qwak utilizes this token initially to set up a group administrator for a Qwak-generated project and subsequently used this dedicated token for operations.
You have several options to tailor how build dependencies are resolved:
- Python Repositories: Choose specific Python repositories from your JFrog account to resolve dependencies. These selected repositories will be incorporated into a virtual repository created by Qwak, named “qwak-python-dependencies-virtual.” This is the only repository Qwak uses for Python dependency resolution. Additionally, selecting the “allow external Python dependencies” option will include “pypi.org” in the virtual repository, allowing for the resolution of dependencies from outside JFrog.
- HuggingFace Repositories: Opt to use an existing HuggingFace repository or let Qwak create one on your behalf. This ensures HuggingFace model dependencies are resolved through the Artifactory proxy instead of directly from HuggingFace, enhancing control and security.
The JFrog account will include the creation of several components:
qwakproject: This is a new project initiated and managed by Qwak. All resources related to Qwak will be organized under this project- Group Admin: A group Role will be established with admin privileges on the project.
qwak-python-dependeciesVirtual repository: This repository serves as the centralized location for resolving Python dependencies.qwak-huggingface-proxy-remote: If ‘create HuggingFace proxy’ selected – a hugging face remote repository will be created in the Qwak project – used as cache for all used HuggingFace ML models.
Overview of Repository Structure in JFrog Artifactory for Qwak Builds
The layout of a Qwak build within JFrog Artifactory is organized as follows:
├── huggingface-remote (single remote repo to HuggingFace. Not necessarily generated by Qwak)
├── pypi-remote/pypi-private-repos (Not necessarily generated by Qwak)
├── qwak-python-dependencies-virtual (Qwak generated virtual repository)
├── …
├── qwak-<Qwak-Project>-artifact-local
├── qwak-<Qwak-Project>-dataset-local
├── qwak-<Qwak-Project>-docker-local
├── <Qwak-Model>
├── <Qwak-Build-ID> (Docker artifact)
├── manifest.json
├── …
For each project, three specific repositories are generated:
qwak-<Qwak-Project>-artifact-local: a generic repository, hosting the artifacts produced by the build.qwak-<Qwak-Project>-dataset-local: a generic repository, containing the dataset artifacts logged by the user during the build process usingqwak.log_data.qwak-<Qwak-Project>-docker-local: A Docker repository that stores the serving images, which are the final output of the build process. These images are used for deployments.
Each model and build is contained within its distinct folder under the respective repository.
Scanning for Vulnerabilities in HuggingFace Models
During the build process, Qwak retrieves HuggingFace models through Artifactory. Each model cached in the remote repository is scanned by JFrog Xray, which not only checks for vulnerabilities but also examines the licensing of the models. This comprehensive scan ensures that every model meets high security standards. Moreover, any policies and watches configured in JFrog are respected by the integration, ensuring consistent policy enforcement and security posture.
Please note that Qwak provides only a summary of the vulnerability scans conducted by JFrog Xray. For detailed insights, you are encouraged to click the “Scan results” button. This action will redirect you to the comprehensive scan report available on JFrog’s platform.
