Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Option to define a public URL for share link #2907

Closed
Joe136 opened this issue Jun 23, 2023 · 9 comments
Closed

Option to define a public URL for share link #2907

Joe136 opened this issue Jun 23, 2023 · 9 comments
Assignees
Labels
community enhancement New feature or request

Comments

@Joe136
Copy link

Joe136 commented Jun 23, 2023

In short: I want to setup docker hub minio/minio behind a reverse proxy. The "share link" feature will use 127.0.0.1:9000 but I want something like s3.chat.example.org with https://. This is also an explanation of the problems in the issues #1205 #1569 #1632 .

After a while of understanding how the domain of the url is resolved in presignedUrl and share link I recognized that the MinIO S3-Api server will use the connection info to create presignedURLs. The Console uses http://127.0.0.1:9000 to connect to the S3-Api and therefore gets a http://127.0.0.1:9000/... share link. When using the MINIO_SERVER_URL env var, the S3-Api will return <MINIO_SERVER_URL>/... as presignedUrl / share link.

By default, the Console uses "localhost:<server port>" to connect to its own MinIO. The env var MINIO_SERVER_URL can be used to change the target MinIO.

When hosting MinIO behind a reverse proxy like NginX and Traefik, the proxy will manage TLS, domain resolution and path redirects. Services behind the proxy can connect to each other by hostnames instead of FQDN. They also may use unencrypted connections (http://) between each other. If a service must send a full url to the user's Browser, the service must use the proper public domain of the proxy (e.g. chat.example.org).

Expected Behavior

When the Console generates a share link, the share link must have the public url in the form https://<public domain>/... (e.g. https://s3.chat.example.org/...). The users of the Console can use the share link to download the file from MinIO Server, the proxy will forward the request to MinIO.

The connection from Console to S3-Api can use (un)encrypted localhost for faster and more energy efficient communication (see 3rd drawback in next block).

Current Behavior

The Console generates share links based on the connection info, this means the share link will have the form http://127.0.0.1:9000/... . I assume this is what the S3-Api returned as presignedUrl. Users can't use the share links because the links are localhost; more precisely: they are not related to the proxy (in any form).

A workaround is to set MINIO_SERVER_URL to the public url #1205 #1569 #1632 (e.g. MINIO_SERVER_URL="https://s3.chat.example.org"), then the generated share links will have the proper form https://s3.chat.example.org/... .

This has the consequence, that the Console uses the proxy to communicate to the S3-Api.
This has many drawbacks:

  1. the proxy must manage the Console <--> S3-Api in addition to user requests
    • the proxy has to manage the requests from the users but not the internal communication
  2. it depends on a working proxy
    • if the proxy/load balancer is overloaded, the communication will fail
    • if the proxy has ANY problems, the communication will fail
  3. the communication Console <--> proxy <--> S3-Api is slower and requires more energy (increases costs)
    • the TLS encryption and the extra processing of the proxy
  4. certificates must be publicy created (letyencrypt) or shared between proxy and Console
    • in local development, letsencrypt is not an option
    • creating accurate self-signed certificates and sharing between proxy and Console is not practical

In addition to this workaround, when assigning the IP in /etc/hosts like 172.25.0.3 s3.chat.example.org , the Console tries to connect to port 443 with TLS, but MinIO is listening on port 9000 without TLS. This don't work because nothing is listening on port 443, at least not MinIO and not with TLS unless configured.

Possible Solution

  1. setup MinIO Server to listen on port 443, enable TLS, setup certificates, assign public url to this container (/etc/hosts)
  2. add a second reverse proxy that resolves the url and handles communication (also with cert and TLS)
    • this relieves the primary load balancer; and reduces proxy errors
  3. add an option in MinIO Console that replaces the urls that point to the MinIO Server (somehow similar to MINIO_BROWSER_REDIRECT_URL)
  4. add an option in MinIO Server that assigns the public url in presignedUrl instead of using the connection info

Personally I prefer the third solution.

Solution 2:

version: "3.9"
volumes:
   nginx-certs:
services:
  nginx:
    image: nginx
    # see: https://min.io/docs/minio/linux/integrations/setup-nginx-proxy-with-minio.html
    volumes:
      nginx-certs:/etc/letsencrypt
  backend-nginx:
    image: nginx
    hostname: s3.chat.example.org
    # Setup contains only config to connect to MinIO S3-Api
    # see: https://min.io/docs/minio/linux/integrations/setup-nginx-proxy-with-minio.html
    volumes:
      nginx-certs:/etc/letsencrypt
  minio:
    image: minio/minio
    hostname: minio
    command: server /data
    environment:
      MINIO_ROOT_USER: minioadmin
      MINIO_ROOT_PASSWORD: minioadmin
      MINIO_CONSOLE_ADDRESS: ":9001"
      MINIO_SERVER_URL: "https://s3.chat.example.org"
      MINIO_BROWSER_REDIRECT_URL: "https://chat.example.org/minio-console"
    volumes:
      - "./data/minio:/data"

Steps to Reproduce (for bugs)

Like in #1632 (comment)

Short form:

version: "3.9"
services:
  nginx:
    image: nginx
    # see: https://min.io/docs/minio/linux/integrations/setup-nginx-proxy-with-minio.html
  minio:
    image: minio/minio
    hostname: minio
    # hostname: s3.chat.example.org   # this will not work
    command: server /data
    environment:
      MINIO_ROOT_USER: minioadmin
      MINIO_ROOT_PASSWORD: minioadmin
      MINIO_CONSOLE_ADDRESS: ":9001"
      MINIO_BROWSER_REDIRECT_URL: "https://chat.example.org/minio-console"
    volumes:
      - "./data/minio:/data"

Context

Useful when using the official minio/minio docker image and running both MinIO and MinIO Console behind a reverse proxy with URL rewriting. When the Console should be accessible to users or coworkers. Reducing computational overhead and necessary configuration for this use case.

Regression

No

Your Environment

Docker compose: Docker hub version minio/minio:RELEASE.2023-06-19T19-52-50Z
OS: Ubuntu 20.04

@cesnietor
Copy link
Collaborator

this would need to be considered as part of #922 solution. @kannappanr

@cesnietor cesnietor self-assigned this Jul 24, 2023
@cesnietor cesnietor added enhancement New feature or request and removed triage labels Jul 24, 2023
@dberardo-com
Copy link

is it possible to access buckets and folders which already have anonymous policies from just any domain ?

say that i have 2 domains from which i can access minio buckets via a reverse proxy, would it be possible to download a file, say: "bucket/test/filename.txt" by just going through:

"https://anydomain.iwant/bucket/test/filename.txt

if i do this, then the request return a index.html page from minio ...

@allanrogerr
Copy link
Contributor

<< @cesnietor dev notes: implement without introducing extra env variables >>

@ondrejsika
Copy link

I have the same issue.

@stevapple
Copy link

Also want MINIO_DOMAIN (aka DNS-style bucket URL) support here.

@james70s
Copy link

james70s commented Apr 3, 2024

minio:
command: server /data --console-address "0.0.0.0:9001"
environment:
MINIO_ROOT_USER: minioadmin
MINIO_ROOT_PASSWORD: minioadmin
MINIO_SERVER_URL: "http://your_url.com:9000"
volumes:
- "./data/minio:/data"

@ngowuys
Copy link

ngowuys commented Apr 5, 2024

Hope there will be a solution for this, I don't really want to open 9000 port and set MINIO_SERRVER_URL=http://your_url.com:9000, since Minio is running behind NGINX already.

@cesnietor
Copy link
Collaborator

Hi, the functionality of this will be changed as mentioned in here #3255 PR already in review. Feel free to express any concerns.

@cesnietor
Copy link
Collaborator

this is merged and there will be a release soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
community enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

8 participants