Skip to content
GitLab
Projects Groups Topics Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Register
  • Sign in
  • G gitlab-ce
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
  • Issues 0
    • Issues 0
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 1
    • Merge requests 1
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Metrics
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • nexedinexedi
  • gitlab-ce
  • Repository
  • gitlab-ce
  • internal
  • api
  • api.go
Find file BlameHistoryPermalink
  • Stan Hu's avatar
    Add Azure Blob Storage support · aa5192b4
    Stan Hu authored Aug 17, 2020
    This merge request introduces a client for Azure Blob Storage in
    Workhorse. Currently customers wanting to use Azure Blob Storage have to
    set up a Minio Gateway
    (https://docs.gitlab.com/charts/advanced/external-object-storage/azure-minio-gateway.html),
    which isn't ideal because it requires customers to maintain their own
    proxy server for Azure. We have a number of customers who want native
    support for Azure Blob Storage.
    
    Unlike AWS and Google, Azure needs to use an Azure client inside
    Workhorse to support direct uploads. Using standard HTTP transfers with
    pre-signed URLs with the Azure Put Blob API
    (https://docs.microsoft.com/en-us/rest/api/storageservices/put-blob)
    doesn't work because Azure doesn't support chunked transfer encoding.
    However, Azure does support uploading files in segments via the Put
    Block and Put Block List API
    (https://docs.microsoft.com/en-us/rest/api/storageservices/put-block),
    but this requires an Azure client that can speak this API.
    
    Instead of embedding the Microsoft Azure client directly, we use the Go
    Cloud Development Kit (https://godoc.org/gocloud.dev/blob) to make it
    easier to add other object storage providers later. For example, GitLab
    Rails might return this JSON payload in the
    `/internal/uploads/authorize` call:
    
    ```json
    {
       "UseWorkhorseClient":true,
       "ObjectStorage":{
          "Provider":"AzureRM",
          "GoCloudConfig":{
             "URL":"azblob://test-bucket"
          }
       }
    }
    ```
    
    The `azblob` scheme is managed by the Go Cloud `URLMux`
    (https://godoc.org/gocloud.dev/blob#URLMux).
    
    Converting our existing S3 client with Go Cloud should be done later
    (https://gitlab.com/gitlab-org/gitlab-workhorse/-/issues/275).
    
    This changes requires
    https://gitlab.com/gitlab-org/gitlab/-/merge_requests/38882 to work.
    
    Omnibus configuration changes are in
    https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/4505.
    
    Part of https://gitlab.com/gitlab-org/gitlab/-/issues/25877
    aa5192b4
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7