Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
slapos slapos
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Labels
    • Labels
  • Merge requests 122
    • Merge requests 122
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Operations
    • Operations
    • Environments
  • Analytics
    • Analytics
    • CI/CD
    • Repository
    • Value Stream
  • Snippets
    • Snippets
  • Members
    • Members
  • Activity
  • Graph
  • Jobs
  • Commits
Collapse sidebar
  • nexedi
  • slaposslapos
  • Merge requests
  • !1498

Merged
Created Dec 29, 2023 by Kirill Smelkov@kirrOwner

format-json, test: Don't force ASCII

  • Overview 4
  • Commits 1
  • Changes 2

Currently, due to ensure_ascii=True default of json.dumps, we are insisting on our JSON schemas to be ascii-only and all other characters to be represented by \uxxxx escapes. So far this was not problematic as all our schemas contains only ASCII characters, but upcoming ors-amarisoft changes want to use e.g. "→" symbol:

https://lab.nexedi.com/kirr/slapos/blob/b51f5523/software/ors-amarisoft/software.cfg.json#L15

which currently results in failure of json-schema test:

FAIL: test_ors-amarisoft_software_cfg_json_format (slapos.test.test_json_schema.TestJSONSchemaValidation)
...

First differing element 14:
'      "title": "\\u2192  eNB/gNB | Radio Unit",'
'      "title": "→  eNB/gNB | Radio Unit",'

And in general, in 2023 I think there is no reason to insist on our schemas to be ASCII-only: say if one wants to describe something about "α" parameter. It would be good to use that α character directly and seeing it in the editor, instead of using escapes all the time.

As indicated by below stackoverflow answer "JSON spec requires UTF-8 support by decoders": https://stackoverflow.com/a/594881/9456786 , and indeed checking JSON specification also confirms that by default JSON decoders shall use UTF-8:

https://datatracker.ietf.org/doc/html/rfc7159#section-8.1

This way, I think, we can switch to UTF-8 safely.

/cc @jerome, @lu.xu

Assignee
Assign to
Reviewer
Request review from
None
Milestone
None
Assign milestone
Time tracking
Source branch: y/json-unicode
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7