Skip to content

UFAL/Download instructions for command line don't work #1210

Closed
dataquest-dev/DSpace
#1252

Description

@milanmajchrak

Original issue: ufal/clarin-dspace#1130 (comment)

Why curl {} brace expansion cannot be used for download commands

Background

The original requirement was to use curl's URL globbing ({}) for compact download commands:

curl -o "#1" "https://server/api/.../handle/123/42{/file1.txt,/file2.txt}"

After thorough investigation, this approach is not viable. Here's why:

Problem 1: Raw (non-encoded) filenames in {} produce invalid HTTP requests

Filenames with spaces, diacritics, or special characters (Médiá (3).jfif) placed directly in {} result in curl sending raw bytes in the URL path. This violates HTTP spec (RFC 7230) — the server returns 400 Bad Request.
If you are trying to use this CURL on windows you must have the right encoding in you terminal.
You find it out using the command chcp

Problem 2: Percent-encoded filenames in {} produce wrong output filenames

curl -o "#1" "url/{M%C3%A9di%C3%A1.jfif,simple.txt}"

The #1 variable is substituted with the literal text from {} — including percent sequences. The file is saved as M%C3%A9di%C3%A1.jfif instead of Médiá.jfif. curl does not URL-decode #1. From the official documentation: "curl does not attempt to decode %-sequences (yet) in the provided file name". There is no --url-decode or similar option.

Problem 3: -J (Content-Disposition) fails on Windows

curl -OJ "url/{M%C3%A9di%C3%A1.jfif,simple.txt}"

The -J flag reads the filename from the server's Content-Disposition header. However, curl on Windows fails with "Invalid argument" when the header contains non-ASCII characters (diacritics, CJK, etc.).

Solution

Use separate -o "filename" "url" pairs for each file:

curl -o "Médiá (3).jfif" "https://server/.../M%C3%A9di%C3%A1%20%283%29.jfif" -o "data.csv" "https://server/.../data.csv"
  • -o receives the real filename (with diacritics) directly from the shell — works on all OS
  • The URL is properly percent-encoded — valid HTTP
  • Works on Windows, Linux, and macOS for any filename

Activity

  1. Paurikova2 commented on Feb 18, 2026

    @Paurikova2

    @milanmajchrak

    THIS IS IMPORTANT

    There is unexising url.
    Correct url> http://localhost:4000/bitstream/123456789/2-5912/0/test.jfif

    And now there is problem:

    The Core Problem

    In DSpace 5, everything ran on one port (8080).
    In DSpace 7+, the system is split:Port 4000 (Frontend): Just a JavaScript (Angular) app. It doesn't have the files.Port 8080 (Backend): The REST API where the files actually live.

    Why Browser works vs. curl fails

    Browser: Loads the Frontend $\rightarrow$ Runs JavaScript $\rightarrow$ Automatically fetches the file from the Backend.
    Curl: Only downloads what is at the URL. Since the Frontend is just an "empty shell," you get 404 or HTML, because curl cannot run the JavaScript needed to find the actual file.

    Solutions for the next developer1.

    1. Call the Backend directly (Recommended)Identify the internal UUID of the bitstream and use the API endpoint:curl -L http://localhost:8080/server/api/core/bitstreams//content -o test.jfif
    2. Find the "True" URLOpen Browser DevTools (F12) > Network tab.Refresh the page and find the request for the file.Use the Request URL found there (it will likely be port 8080).
    3. Fix via Proxy (If port 4000 must work)If you need the frontend to serve these files, ensure the Express server is configured to proxy /bitstream requests to the backend:JavaScript// Example in Express
      app.use('/bitstream', createProxyMiddleware({ target: 'http://localhost:8080' }));

    Maybe helpfull

    dataquest-dev/DSpace#1048

  2. added 2 commits that reference this issue on Jul 2, 2026
    e9792ad
    7c551ea
  3. added 2 commits that reference this issue on Jul 14, 2026
    dc0be47
    2e2e523
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions