Comment by mdaniel

3 years ago

Tested on python what? I was curious to see what error that produced, figuring it would be some whitespace due to the difference between the list items, but using the yamlized python that I had lying around, it did the sane thing:

    PATH=$HOMEBREW_PREFIX/opt/ansible/libexec/bin:$PATH
    pip list | grep -i yaml
    python -V
    python <<'DOIT'
    from io import StringIO
    import yaml
    print(yaml.safe_load(StringIO(
    '''
        {
            "list": [
                {},
                    {}
            ]
        }
    ''')))
    DOIT

produces

    PyYAML                6.0
    Python 3.10.1
    {'list': [{}, {}]}

15 comments

mdaniel

Reply
hoherd  3 years ago

With leading tabs it does not work.

  $ sed 's/\t/--->/g' break-yaml.json
  --->{
  --->--->"list": [
  --->--->--->{},
  --->--->--->{}
  --->--->]
  --->}
  $ jq -c . break-yaml.json
  {"list":[{},{}]}
  $ yaml-to-json.py break-yaml.json
  ERROR: break-yaml.json could not be parsed
  while scanning for the next token
  found character '\t' that cannot start any token
    in "break-yaml.json", line 1, column 1
  $ sed 's/\t/    /g' break-yaml.json | yaml-to-json.py
  {"list": [{}, {}]}

remram  3 years ago

Edited with string escapes, the tab didn't make it through HN.

The error from PyYaml 5.3.1:

    yaml.scanner.ScannerError: while scanning for the next token
    found character '\t' that cannot start any token
      in "<unicode string>", line 4, column 1

  • notreallyserio  3 years ago

    If it continues to be hard to share, I suggest encoding it as a base64 string so folks can decode it into a file with exactly the right contents.

    • jwilk  3 years ago

      Not base64, but this should be easy to reproduce:

        $ printf '{\n\t"list": [\n\t\t{},\n\t\t{}\n\t]\n}\n' > test.json

  $ jq < test.json 
  {
    "list": [
      {},
      {}
    ]
  }

  $ yamllint test.json 
  test.json
    2:1       error    syntax error: found character '\t' that cannot start any token (syntax)

  • ak217  3 years ago

    Thanks, I'm finally able to reproduce this.

    It would be great if instead of the histrionic message on CPAN (which amusingly accuses others of "mass hysteria"), the author would just say "YAML documents can't start with a tab while JSON documents can, making JSON not a strict subset of YAML".

    The YAML spec should be updated to reflect this, but I wonder if a simple practical workaround in YAML parsers (like replacing each tab at the beginning of the document with two spaces before feeding it to the tokenizer) would be sufficient in the short term.

    • ingy  3 years ago

      > "YAML documents can't start with a tab while JSON documents can, making JSON not a strict subset of YAML"

      But YAML can start with tabs. Tabs are allowed as separating whitespace in most of the spec productions but are not allowed as indentation. Even though those tabs look like indentation, the spec productions don't interpret them as such.

      See my comment above and esp see https://play.yaml.io/main/parser?input=CXsKCQkibGlzdCI6IFsKC...

      Note: the YAML spec maintainers (I am one) have identified many issues with YAML which we are actively working on, but (somewhat surprisingly) we have yet to find a case where valid JSON is invalid YAML 1.2.

      1 reply →