musicxml2ly produces unexpected EOF in .ly file

[Victor Rouanet]

Good morning,

I'm having an issue with musicxml2ly, installed with a manually
downloaded version on lilypond 2.19.82.

I exported a musicxml file from Musescore, then converted it to lilypond
source with musicxml2ly without error. However, when I try to compile it
with lilypond, I get the following error: "syntax error, unexpected end
of input, expecting \header". Here is the full output:
https://hastebin.com/raw/navufewufu

And the musicxml file content: https://hastebin.com/jafanosumu.xml

This error happened with any musicxml file I tried to convert from
Musescore.

After some research, I found this topic on the lilypond-user list
http://lilypond.1069038.n5.nabble.com/XML-to-ly-and-Lilypond-again-td203101.html
which looks like exactly the same.
The suggested workaround (replacing
split_string_and_preserve_doublequoted_substrings by string.split in
musicexp.py:139) makes it work, but I'm wondering if I'm doing something
wrong.

Victor

[Federico Bruni]

Il giorno ven 24 ago 2018 alle 11:33, Victor Rouanet
<"victor.rouanet"@gmx.fr> ha scritto:
> Good morning,
>
> I'm having an issue with musicxml2ly, installed with a manually
> downloaded version on lilypond 2.19.82.
>
> I exported a musicxml file from Musescore, then converted it to
> lilypond source with musicxml2ly without error. However, when I try
> to compile it with lilypond, I get the following error: "syntax
> error, unexpected end of input, expecting \header". Here is the full
> output: https://hastebin.com/raw/navufewufu
>
> And the musicxml file content: https://hastebin.com/jafanosumu.xml
>
> This error happened with any musicxml file I tried to convert from
> Musescore.
>
> After some research, I found this topic on the lilypond-user list
> http://lilypond.1069038.n5.nabble.com/XML-to-ly-and-Lilypond-again-td203101.html
> which looks like exactly the same.
> The suggested workaround (replacing
> split_string_and_preserve_doublequoted_substrings by string.split in
> musicexp.py:139) makes it work, but I'm wondering if I'm doing
> something wrong.
>
>

Which operating system are you using?

On Linux, I don't get any EOF error.
Find attached the .ly file generated by musicxml2ly version 2.19.82.
There's a wrong input though:

$ /usr/bin/lilypond jafanosumu.ly
GNU LilyPond 2.19.82
Processing `jafanosumu.ly'
Parsing...
jafanosumu.ly:30:17: error: syntax error, unexpected '^'
\tempo 4=80
^\markup{ \bold {Moderato} } | % 1
jafanosumu.ly:30:18: error: markup outside of text script or \lyricmode
\tempo 4=80 ^
\markup{ \bold {Moderato} } | % 1
jafanosumu.ly:32:34: error: syntax error, unexpected '_'
\mark \markup { \box { A } }
_\markup{ \italic {sempre forte} } | %
2
jafanosumu.ly:32:35: error: markup outside of text script or \lyricmode
\mark \markup { \box { A } } _
\markup{ \italic {sempre forte} } | %
2
jafanosumu.ly:33:51: error: syntax error, unexpected '^'
\stemDown e4 \stemDown c4 \stemUp g2 \breathe
^\markup{ \italic
jafanosumu.ly:33:52: error: markup outside of text script or \lyricmode
\stemDown e4 \stemDown c4 \stemUp g2 \breathe ^
\markup{ \italic
jafanosumu.ly:41:5: error: errors found, ignoring music expression

<<
fatal error: failed files: "jafanosumu.ly"

[Federico Bruni]

Victor, please keep the list in Cc.

I was using lilypond packaged by Fedora, which uses python2 installed
on the system (currently 2.7.15).
I've now installed 2.19.82 from lilypond.org, which uses the bundled
python .
There's something wrong:

$ file jafanosumu.ly
jafanosumu.ly: data

There's some bad encoding. See attached file.

$ ls -lh ~/.local/lilypond/usr/bin/python*
-rwxr-xr-x 2 fede fede 5,1K 24 giu 14.11
/home/fede/.local/lilypond/usr/bin/python
-rwxr-xr-x 2 fede fede 5,1K 24 giu 14.11
/home/fede/.local/lilypond/usr/bin/python2.4

It's just my installation?
This release had also some problems in the PDF manuals...

Il giorno ven 24 ago 2018 alle 13:19, Victor Rouanet
<"victor.rouanet"@gmx.fr> ha scritto:
> I use debian 9 (stretch). I tried to redownload the raw XML from the
> link I provided (with wget) and got the same error.
>
> Do you know what python version were used by musicxml2ly when you
> tried?
>
> Yes, I was initially planning to report an issue about this wrong
> output, but I upgraded lilypond before and had to edit the
> musicexp.py again
>
>
> Le 24/08/2018 à 12:02, Federico Bruni a écrit :
>>
>>
>> Il giorno ven 24 ago 2018 alle 11:33, Victor Rouanet
>> <"victor.rouanet"@gmx.fr> ha scritto:
>>> Good morning,
>>>
>>> I'm having an issue with musicxml2ly, installed with a manually
>>> downloaded version on lilypond 2.19.82.
>>>
>>> I exported a musicxml file from Musescore, then converted it to
>>> lilypond source with musicxml2ly without error. However, when I
>>> try to compile it with lilypond, I get the following error:
>>> "syntax error, unexpected end of input, expecting \header". Here
>>> is the full output: https://hastebin.com/raw/navufewufu
>>>
>>> And the musicxml file content: https://hastebin.com/jafanosumu.xml
>>>
>>> This error happened with any musicxml file I tried to convert from
>>> Musescore.
>>>
>>> After some research, I found this topic on the lilypond-user list
>>> http://lilypond.1069038.n5.nabble.com/XML-to-ly-and-Lilypond-again-td203101.html
>>> which looks like exactly the same.
>>> The suggested workaround (replacing
>>> split_string_and_preserve_doublequoted_substrings by string.split
>>> in musicexp.py:139) makes it work, but I'm wondering if I'm doing
>>> something wrong.
>>>
>>>
>>
>> Which operating system are you using?
>>
>> On Linux, I don't get any EOF error.
>> Find attached the .ly file generated by musicxml2ly version 2.19.82.
>> There's a wrong input though:
>>
>> $ /usr/bin/lilypond jafanosumu.ly
>> GNU LilyPond 2.19.82
>> Processing `jafanosumu.ly'
>> Parsing...
>> jafanosumu.ly:30:17: error: syntax error, unexpected '^'
>> \tempo 4=80
>> ^\markup{ \bold {Moderato} } | % 1
>> jafanosumu.ly:30:18: error: markup outside of text script or
>> \lyricmode
>> \tempo 4=80 ^
>> \markup{ \bold {Moderato} } | % 1
>> jafanosumu.ly:32:34: error: syntax error, unexpected '_'
>> \mark \markup { \box { A } }
>> _\markup{ \italic {sempre forte} } |
>> % 2
>> jafanosumu.ly:32:35: error: markup outside of text script or
>> \lyricmode
>> \mark \markup { \box { A } } _
>> \markup{ \italic {sempre forte} } |
>> % 2
>> jafanosumu.ly:33:51: error: syntax error, unexpected '^'
>> \stemDown e4 \stemDown c4 \stemUp g2 \breathe
>> ^\markup{ \italic
>> jafanosumu.ly:33:52: error: markup outside of text script or
>> \lyricmode
>> \stemDown e4 \stemDown c4 \stemUp g2 \breathe ^
>> \markup{ \italic
>> jafanosumu.ly:41:5: error: errors found, ignoring music expression
>>
>> <<
>> fatal error: failed files: "jafanosumu.ly"
>>
>>
>>
>

[Victor Rouanet]

Le 26/08/2018 à 12:56, Federico Bruni a écrit :
> Victor, please keep the list in Cc.

Sorry, newbie error :-)

> I was using lilypond packaged by Fedora, which uses python2 installed
> on the system (currently 2.7.15).
> I've now installed 2.19.82 from lilypond.org, which uses the bundled
> python .
> There's something wrong:
>
> $ file jafanosumu.ly
> jafanosumu.ly: data
>
> There's some bad encoding. See attached file.
>
> $ ls -lh ~/.local/lilypond/usr/bin/python*
> -rwxr-xr-x 2 fede fede 5,1K 24 giu 14.11
> /home/fede/.local/lilypond/usr/bin/python
> -rwxr-xr-x 2 fede fede 5,1K 24 giu 14.11
> /home/fede/.local/lilypond/usr/bin/python2.4
>
> It's just my installation?

Same result for me. According to Phil Homes in the lilypond-user topic,
the shlex module (used by musicxml2ly in utilities.py) didn't support
unicode input prior to Python 2.7.3. That would explain why the
conversion only fails with manually downloaded version of lilypond,
which bundles python 2.4.5.
When I hand edit the musicxml2ly script, forcing it to use the system
python version (2.7.13), the result file looks well encoded.

$ file -i jafanosumu_converted_using_python2.*
jafanosumu_converted_using_python2.4.5.ly: application/octet-stream;
charset=binary
jafanosumu_converted_using_python2.7.13.ly: text/plain; charset=us-ascii

[Victor Rouanet]

Le 26/08/2018 à 15:15, Victor Rouanet a écrit :
> Same result for me. According to Phil Homes in the lilypond-user
> topic, the shlex module (used by musicxml2ly in utilities.py) didn't
> support unicode input prior to Python 2.7.3.
According to David Wright*
I misread.

[David Kastrup]

Victor Rouanet <***@gmx.fr> writes:

> Le 26/08/2018 à 15:15, Victor Rouanet a écrit :
>> Same result for me. According to Phil Homes in the lilypond-user
>> topic, the shlex module (used by musicxml2ly in utilities.py) didn't
>> support unicode input prior to Python 2.7.3.
> According to David Wright*
> I misread.

So we should upgrade Python in GUB?

--
David Kastrup

[Federico Bruni]

Il giorno dom 26 ago 2018 alle 16:12, David Kastrup <***@gnu.org> ha
scritto:
> Victor Rouanet <***@gmx.fr> writes:
>
>> Le 26/08/2018 à 15:15, Victor Rouanet a écrit :
>>> Same result for me. According to Phil Homes in the lilypond-user
>>> topic, the shlex module (used by musicxml2ly in utilities.py)
>>> didn't
>>> support unicode input prior to Python 2.7.3.
>> According to David Wright*
>> I misread.
>
> So we should upgrade Python in GUB?
>

https://github.com/gperciva/gub/pull/6

I hope more people will be able to run GUB successfully. Only Phil can
run it at the moment?
I look forward to Knut's patch.

[James Lowe]

Knut


On 27/08/18 16:14, Federico Bruni wrote:
>
>
> Il giorno dom 26 ago 2018 alle 16:12, David Kastrup <***@gnu.org> ha
> scritto:
>> Victor Rouanet <***@gmx.fr> writes:
>>
>>>  Le 26/08/2018 à 15:15, Victor Rouanet a écrit :
>>>>  Same result for me. According to Phil Homes in the lilypond-user
>>>>  topic, the shlex module (used by musicxml2ly in utilities.py) didn't
>>>>  support unicode input prior to Python 2.7.3.
>>>  According to David Wright*
>>>  I misread.
>>
>> So we should upgrade Python in GUB?
>>
>
> https://github.com/gperciva/gub/pull/6
>
> I hope more people will be able to run GUB successfully. Only Phil can
> run it at the moment?
> I look forward to Knut's patch.
>

Your GUB patch has been sitting on the 'to push' list for a few weeks now.

Can you make the pull request please to https://github.com/gperciva/gub
so that Phil can merge your changes into GUB?

Thanks

James

[Paul Hodges]

I was planning to report this.  What's happening is that strings are being converted to 16-bit Unicode instead of UTF8, and so they are interspersed with nulls.  If you remove the nulls with a capable editor, then that issue goes away (you can do it with Frescobaldi, but it's non-intuitive - copy part of a string into the replace dialogue, the nulls appear as spaces, delete everything except one null then replace with nothing throughout the file).


Other issues:


The line

        \mergeDifferentlyDottedOn\mergeDifferentlyHeadedOn
which occurs in the score section for each stave causes the bar-checks to fail (I don't get why, but that's how it is), so it would be nice to be able to turn that off.


Also, the score is full of explicit \stemUp and \stemDown commands, which it would also be good to be able to omit.


Finally, I don't know why, but the conversion fails for me on any XML generated by Sibelius 7.5 (the version I have) or MuseScore (current).  In both cases, I can read them into Dorico which can then rewrite them in a way that the Lilypond converter reads successfully.


Paul


From: Victor Rouanet <***@gmx.fr>
To: <bug-***@gnu.org>
Sent: 24/08/2018 10:33 AM
Subject: musicxml2ly produces unexpected EOF in .ly file

Good morning,

I'm having an issue with musicxml2ly, installed with a manually
downloaded version on lilypond 2.19.82.

I exported a musicxml file from Musescore, then converted it to lilypond
source with musicxml2ly without error. However, when I try to compile it
with lilypond, I get the following error: "syntax error, unexpected end
of input, expecting \header". Here is the full output:
https://hastebin.com/raw/navufewufu

And the musicxml file content: https://hastebin.com/jafanosumu.xml

This error happened with any musicxml file I tried to convert from
Musescore.

After some research, I found this topic on the lilypond-user list
http://lilypond.1069038.n5.nabble.com/XML-to-ly-and-Lilypond-again-td203101.html
which looks like exactly the same.
The suggested workaround (replacing
split_string_and_preserve_doublequoted_substrings by string.split in
musicexp.py:139) makes it work, but I'm wondering if I'm doing something
wrong.

Victor


_______________________________________________
bug-lilypond mailing list
bug-***@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-lilypond

[Federico Bruni]

Il giorno ven 24 ago 2018 alle 12:11, Paul Hodges <***@cassland.org> ha
scritto:
> I was planning to report this. What's happening is that strings are
> being converted to 16-bit Unicode instead of UTF8, and so they are
> interspersed with nulls. If you remove the nulls with a capable
> editor, then that issue goes away (you can do it with Frescobaldi,
> but it's non-intuitive - copy part of a string into the replace
> dialogue, the nulls appear as spaces, delete everything except one
> null then replace with nothing throughout the file).
>

Please let us know your operating system and LilyPond version.
Can you also attach the problematic .ly file?
Thanks