Bug in new YAML parser?

0 votes
asked Jan 27, 2021 in Bug by boldandbusted (280 points)

Howdy. I was just kicking the tires on the new YAML feature. I gave it a challenge: can it use a moderately complex Kubernetes Helm Chart manifest without major changes? Sadly, no.

My preparation process:

* Go to a random Kubernetes Helm chart (https://github.com/prometheus-operator/kube-prometheus.git), and grabbed https://github.com/prometheus-operator/kube-prometheus.git/manifests/grafana-deployment.yaml

* Validate it in a YAML Validator at https://yamlvalidator.com/

* Put "@startyaml" at the beginning of the file, and "@endyaml" at the end - no other changes.

* Run "plantuml -v' and see:

PlantUML version 1.2021.00 (Sun Jan 10 05:25:05 EST 2021)
(GPL source distribution)
Java Runtime: OpenJDK Runtime Environment
JVM: OpenJDK 64-Bit Server VM
Default Encoding: UTF-8
Language: en
Country: US
 
PLANTUML_LIMIT_SIZE: 4096

Dot version: dot - graphviz version 2.44.1 (20200629.0846)
Installation seems OK. File generation OK

* Then, after renaming the file, run 'plantuml grafana-deployment.plantuml', and see

$ plantuml grafana-deployment.plantuml
java.lang.UnsupportedOperationException:     app.kubernetes.io/component: grafana
        at net.sourceforge.plantuml.yaml.SimpleYamlParser.parseSingleLine(SimpleYamlParser.java:182)
        at net.sourceforge.plantuml.yaml.SimpleYamlParser.parse(SimpleYamlParser.java:68)
        at net.sourceforge.plantuml.yaml.YamlDiagramFactory.createSystem(YamlDiagramFactory.java:70)
        at net.sourceforge.plantuml.PSystemBuilder.createPSystem(PSystemBuilder.java:134)
        at net.sourceforge.plantuml.BlockUml.getDiagram(BlockUml.java:163)
        at net.sourceforge.plantuml.SourceFileReaderAbstract.getGeneratedImages(SourceFileReaderAbstract.java:149)
        at net.sourceforge.plantuml.Run.manageFileInternal(Run.java:538)
        at net.sourceforge.plantuml.Run.processArgs(Run.java:422)
        at net.sourceforge.plantuml.Run.manageAllFiles(Run.java:389)
        at net.sourceforge.plantuml.Run.main(Run.java:198)
I tried to also run it in the online plantuml server here. Happy to provide more info. Thank you for PlantUML! :)
commented Jan 27, 2021 by Martin (8,360 points)
edited Jan 27, 2021 by Martin

The link in the OP didn't work for me (perhaps because I don't have a github user, but I found the YAML at https://github.com/prometheus-operator/kube-prometheus/blob/master/manifests/grafana-deployment.yaml

The first line in the YAML that I found that didn't work was:

@startyaml
app.kubernetes.io/component: grafana
@endyaml 

It is the punctuation ("." and "/") in the LHS that it doesn't like.  Although they work in the RHS.  I think Plantuml is trying to force JSON syntax onto the key - but that's not a YAML requirement.

Replacing with hyphens and it works fine:

Extra info:

I have to say that the YAML.ORG spec is pretty hard to read.  I think the relevant section is "Example 7.11. Plain Implicit Keys" under "7.3.3. Plain Style".

I think the relevant syntax path is:

[133]ns-plain-one-line(block-key)

following it through, it seems to define the key as a same-line string of any printable characters (including space), with just three exceptions: the first character is slightly restricted from being some punctuation, and '#' is slightly restricted by what can come before it, and ':' is slightly restricted by what can come after it.  But pretty much it's 'anything goes' until you hit the ': '.

So I agree with OP that the YAML is valid and would ideally be handled by PlantUML.

commented Jan 28, 2021 by boldandbusted (280 points)
Sorry about the 404'd link - @Martin, you found the right one. :) Thank you for validating my findings, and expanding on them. :)

1 Answer

+1 vote
answered Feb 2, 2021 by plantuml (294,960 points)
Thanks for the report.

This should be fixed in last release V1.2021.1
commented Feb 2, 2021 by boldandbusted (280 points)
Wow, fantastic! Thank you!
commented Feb 3, 2021 by Martin (8,360 points)
edited Feb 3, 2021 by Martin

The OP's chosen example yaml now parses a lot further - up to line #138.

But...we still have "Your data does not sound like YAML data" for this:

The problem is with:

      - configMap:
          name: grafana-dashboards
        name: grafana-dashboards

(Annoyingly Plantuml doesn't allow leading spaces (first yaml element must start in position 1), nor "-" entries at the root (see Example 2.1.  Sequence of Scalars in https://yaml.org/spec/1.2/spec.html), which makes it harder to debug snippets, I could raise a new case for this.)

I have a theory.

Although Plantuml supports "compact nested mapping" for simple maps.

i.e.

@startyaml
A:
  -
    B:C
    D:E
@endyaml

can be compacted as:

@startyaml
A:
  - B:C
    D:E
@endyaml

However, this doesn't extend to more complex mapping.

i.e.

@startyaml
A:
  -
    B:
      C:D
      E:F
@endyaml

doesn't compact properly to:

@startyaml
A:
  - B:
      C:D
      E:F
@endyaml

I hope this helps improve a great feature.

I've opened some Wanted Features for the above:

commented Feb 4, 2021 by plantuml (294,960 points)
We have improved the YAML parser in last beta http://beta.plantuml.net/plantuml.jar

The online server has also been updated, but there might be some caching issue.

Please go on with your tests :-)

Thanks!
commented Feb 5, 2021 by Martin (8,360 points)
edited Feb 5, 2021 by Martin
Fantastic work.  The OP's chosen example now parses!  Looks best in svg due to its size.  (Thanks for the reminder about caching, control-F5 is my new friend.)
commented Feb 24, 2021 by Jesse Adelman

Thank you for making this parse! I do still have problems with the plantuml CLI not parsing the file:

 $ plantuml grafana-deployment.plantuml
java.lang.ClassCastException: class net.sourceforge.plantuml.json.JsonString cannot be cast to class net.sourceforge.plantuml.json.JsonObject (net.sourceforge.plantuml.json.JsonString and net.sourceforge.plantuml.json.JsonObject are in unnamed module of loader 'app')
        at net.sourceforge.plantuml.yaml.SimpleYamlParser.search(SimpleYamlParser.java:257)
        at net.sourceforge.plantuml.yaml.SimpleYamlParser.search(SimpleYamlParser.java:257)
        at net.sourceforge.plantuml.yaml.SimpleYamlParser.search(SimpleYamlParser.java:257)
        at net.sourceforge.plantuml.yaml.SimpleYamlParser.getWorking(SimpleYamlParser.java:225)
        at net.sourceforge.plantuml.yaml.SimpleYamlParser.parseSingleLine(SimpleYamlParser.java:162)
        at net.sourceforge.plantuml.yaml.SimpleYamlParser.parse(SimpleYamlParser.java:68)
        at net.sourceforge.plantuml.yaml.YamlDiagramFactory.createSystem(YamlDiagramFactory.java:70)
        at net.sourceforge.plantuml.PSystemBuilder.createPSystem(PSystemBuilder.java:134)
        at net.sourceforge.plantuml.BlockUml.getDiagram(BlockUml.java:163)
        at net.sourceforge.plantuml.SourceFileReaderAbstract.getGeneratedImages(SourceFileReaderAbstract.java:149)
        at net.sourceforge.plantuml.Run.manageFileInternal(Run.java:538)
        at net.sourceforge.plantuml.Run.processArgs(Run.java:422)
        at net.sourceforge.plantuml.Run.manageAllFiles(Run.java:389)
        at net.sourceforge.plantuml.Run.main(Run.java:198)

Strangely, the same code *does* parse and PNG (truncated output) and SVG diagrams are output when using the PlantUML Online Server at the same version as my CLI plantuml package. How do I know it is the same? I used "cat <filename> | pbcopy" to copy and paste into the Online Server. :) Also, the ASCIIiart output is sadly broken, however - no borders or linking lines.

A suggested feature: Allow raw YAML files to be parsed, without @startyaml/@endyaml stanzas, perhaps with a command-line argument? Then, looping over directories to make diagrams of all the .yaml files becomes trivial and automatable. :)

CLI Version info:

plantuml -version
PlantUML version 1.2021.01 (Tue Feb 02 02:55:08 EST 2021)
(GPL source distribution)
Java Runtime: OpenJDK Runtime Environment
JVM: OpenJDK 64-Bit Server VM
Default Encoding: UTF-8
Language: en
Country: US

PLANTUML_LIMIT_SIZE: 4096

Dot version: dot - graphviz version 2.46.1 (20210213.1702)
Installation seems OK. File generation OK

Cheers!

commented Apr 2, 2021 by boldandbusted (280 points)
So, just tried again with my favorite source (though the version dated changed in March, not the original commit), and it parses! Very nice. Thank you for this feature and for fixing the bugs! :)
...