Synthesia 11 Release Estimate

Try new versions before anyone else!
Always the latest dev version: WindowsmacOSiPadAndroid (or .apk)
Please report comments and bugs!

Your data hasn't disappeared: PC/Mac development previews store their data in a different place. Details here.
JimNYC
Posts: 40

Post by JimNYC »

Nicholas wrote: 10-20-22 3:23 pm After many years of saying there are still many years to go on Synthesia 11, all of a sudden it is looking like there is a good chance that the entirety of beautifully-engraved MusicXML-based sheet music might only take a few more months of integration work instead of more years of from-scratch development.
That's excellent news!! In the past year and a half, I still have yet to find something works as smoothly as Synthesia for custom songs.

If you need any help testing, please don't hesitate to reach out!!!
JimNYC
Posts: 40

Post by JimNYC »

Nicholas wrote: 10-21-22 2:14 pm I haven't tested it yet, but I know the library has extensive support for arbitrarily printing text (lyrics, tempo directions, TAB notation, tuplet beat counts, etc.) in the music. So, even if it's something that doesn't exist yet, it will be easier to add it there than to add it to Synthesia. :anxious:

The code is right on GitHub and the author seems to welcome outside contributions, so it's something I could help maintain, add features to, and fix bugs. (That's probably the most attractive part of the idea. If it were a closed library, I would be stuck with whichever bugs were in there and might have to tell users "sorry, I can't do anything about it". For such an important part of Synthesia, I would never put myself or its users in that position. But because there's ready access to fix things, it should work nicely.)
It looks like fingering is already in Lomse from a previous release. From the change log of version 0.29.0 (current version is 0.30.0 from September):

Version 0.29.0 (11/Mar/2022)
More music notation supported:
Fingering
Pedal marks
All accidental signs
Non-standard key signatures
Full support for cue notes

Improvements in MusicXML support:
Enable edition in MusicXML
Import <staff-lines> elements
Import <defaults> element
Import rest placement on the staff
Import <fingering> elements
Support MusicXML files encoded in UTF-16
Nicholas
Posts: 13144

Post by Nicholas »

That's nice to hear a confirmation.

There will still be the extra step of connecting Synthesia's auxiliary finger hint information to the renderer. Otherwise it would only be able to show hints stored in the MusicXML file itself and not those you add via the in-app interface or via metadata, etc. But I don't anticipate that being too tricky.

Well... maybe. There's an interesting correspondence problem between the falling note blocks and the sheet music anytime you have a proper repeat. There will be two (or more) note blocks that represent the same note in the sheet music... so what should be shown in the sheet music if you assign different finger hints to each block?

It's not even that pathological of a question: if it's the last note in the repeated section before choosing between different endings, you may actually want to use a different finger the second time around to lead into the new section that follows. Hmm. I suppose having something in the falling notes that becomes slightly inexpressible in the sheet music isn't the end of the world. I could probably choose whatever is the easiest answer (the hint on the first note block "win" for the sheet music view, etc.)
JimNYC
Posts: 40

Post by JimNYC »

Does this even exist as a construct in actual sheet music? If not, then I would just take the path of least resistance and assign it the first one (as well as having fingering as an option to display or not). If someone really wanted to express different fingering like your scenario in sheet music, they could always collapse the repeats in music editing software and duplicate the sections instead of repeats. Then it would be easy to assign it different fingering for each block.
Electrode
Posts: 195

Post by Electrode »

I know it's been a while since this post was last updated, but you could do what usually happens in sheet music when alternate fingers are provided. Alternate fingerings are either provided (depending on publisher/edition) by using slash notation or brackets/parentheses.

So, you could express alternate fingering by using 2/3 or 2(3), for example, for a main fingering choice of 2 with the alternate being 3. Of course, this only works for individual changes. For a group of notes with alternate fingering, things might get a little more complicated. That might require something like this, perhaps:

Image
Nicholas
Posts: 13144

Post by Nicholas »

That's interesting. I don't recall seeing the slash/parenthesis notation before. In Behind Bars, finger substitution is shown (on p.309) with a dash between the two fingers... but this isn't really finger substitution either. There isn't any mention there of alternate fingerings.

(I like the footnote on that page, hehe.)
JimNYC
Posts: 40

Post by JimNYC »

Nicholas wrote: 10-20-22 3:23 pm
After many years of saying there are still many years to go on Synthesia 11, all of a sudden it is looking like there is a good chance that the entirety of beautifully-engraved MusicXML-based sheet music might only take a few more months of integration work instead of more years of from-scratch development.

I recently stumbled upon the LenMus project, created by Cecilio Salmerón over even more years than Synthesia. He has (incredibly graciously) open-sourced the sheet-rendering part of his project as a library called Lomse. It's in C++, the license is compatible with Synthesia, and the dependencies are (rather coincidentally) all things that Synthesia already has integrated into its codebase. It uses the same musical font that Synthesia has been using for several years and I've even seen half a dozen references to Behind Bars in the code, so it appears we've even been using the same reference book to make decisions.
Any update on the MusicXML sheet music integration with Loomse? Looking at the development release graph, it doesn't look like there has been any change in the last few months. Is the sheet music integration still looking promising?
Nicholas
Posts: 13144

Post by Nicholas »

There has been a delay for sure, sorry. I was pulled in another direction for half a year and I'm just starting to discover the backlog that has been piling up here in the meantime.

In my time away, I thought about Lomse a lot. I suspect it would still be the most expedient way to get competent sheet music into Synthesia. By a pretty significant margin, too. But from a philosophical (and even practical software delivery) standpoint, the "library" is larger than all of Synthesia combined.

Software continues to eat the world, app sizes double every couple years, the hardware gets faster while the actual experience of using apps steadily declines and this doesn't seem to alarm most people... but it's something that I spend a lot of time thinking about.

Most of the Lomse integration work will be in severing many tendrils from the kraken so that it can fit neatly inside Synthesia's code base. How many unforeseen roadblocks will crop up during that process is anybody's guess. If I had a crystal ball, this would be an easier decision.

The urge to do the Right Thing™ and make it myself from scratch is strong, but you guys have been waiting over a decade at this point and the guilt has mounted to almost the breaking point.
JimNYC
Posts: 40

Post by JimNYC »

Glad to see you are back.

While I understand the urge to do it from scratch allows for complete control, there is no doubt that MusicXML rendering is a huge undertaking judging by the state of other apps & tools. One only needs to read through the discord & GitHub for OSMD to see all the various items that come up just to build a renderer. Dev teams with multiple people seem to constantly be working through issues.

Given that Lomse is open source and is in the right language/structure to integrate, it seems like you get the best of both worlds: a starting point which has the nuts and bolts figured out, plus the ability to alter/customize it as needed. Synthesia is so great with it's ability to play along with each hand, but the sheet music is not really usable as is for practicing something unfamiliar. Any improvement there would be welcomed, especially if it isn't going to require years to implement. :)

I don't think the file size itself matters to anyone, but performance certainly is a factor. Maybe there could even be an option during installation which allows one to choose whether to support MusicXML rendering and if not selected, it is something similar to how it is today in terms of filesize and functionality.
Nicholas
Posts: 13144

Post by Nicholas »

You're probably right.

I only lament that I've gotten through about half of the "hard" parts: the actual primitive rendering (like, drawing individual pixels) in Synthesia's code is very nice and super fast. I paid special attention to making each line in a staff uniform, having the corners of bar lines match up with the staff, not having anything look blurry, and all these other little details that every other notation app (including the major ones) often get wrong... but using Lomse for rendering means not getting to use any of that.

The scarier part is that while Synthesia can now reasonably competently load MusicXML files, Lomse will have to do it again (under the hood) and then the glue code will have to try to reconcile any differences between the results. I can't know for sure, but that feels like it might be a rabbit hole of integration work that takes a lot of effort with nothing really useful being added.

That last bit is the risky unknown that has been keeping me undecided.
JimNYC
Posts: 40

Post by JimNYC »

On the flip side, the path to doing MusicXML rendering from scratch doesn't have to be all or nothing if it's not too daunting. It can be incremental and support a core set of items initially (particularly notes on correct staff, stem direction, ties, etc.). Fingering notation would also be really useful. Other more complicated items in the XML could be supported over time. Anything closer to the real notation would be an improvement users could benefit from.
Chompy
Posts: 182

Post by Chompy »

I just have a question - when will Synthesia 11 ever come out? :?: Sounds like Synthesia 11 is going to be very big, and Synthesia 11 has been worked on since 2020.
Nicholas
Posts: 13144

Post by Nicholas »

JimNYC wrote: 11-14-23 11:38 am... support a core set of items initially (particularly notes on correct staff, stem direction, ties, etc.).
Drawing ties (and slurs more generally) is really the last piece of "new tech" that Synthesia's own sheet rendering component would need before an incremental approach was possible. Otherwise, blindly drawing the note-heads described in the MusicXML would make a tied note appear as separate notes, which would be even less usable.

(I suppose an honorable mention might go to showing more than two beams. The code to draw beams exists, but there's a little bit of work before it could show finer than 16th notes.)
Chompy wrote: 11-15-23 12:49 pm... and Synthesia 11 has been worked on since 2020.
It's more embarrassing than that: the main topic was posted in early 2016, and I'm pretty sure I'd already been talking about "better sheet music" for more than a few years at that point. Searching the forum, it looks like I had started talking seriously about eventually supporting MusicXML files all the way back in 2009. :grimace:
JimNYC
Posts: 40

Post by JimNYC »

So, it sounds like Lomse is out and it's going to be built from the ground up if I had to guess. The posts over a year ago had so much promise so I only hope this is feasible. From my perspective, having notes appear on the correct staff would be a huge improvement from the current musicxml rendering.
Nicholas
Posts: 13144

Post by Nicholas »

JimNYC wrote: 11-18-23 8:19 amSo, it sounds like Lomse is out...
Not necessarily. That bit about ties being the last of the new drawing tech required has been true for at least two years now. (So it wasn't new information or anything.)

Just getting things on the correct staff would be relatively "easy" now in Synthesia's code, too. That could be one of the first small steps, if need be. That said, I do still plan to give Lomse a shot and see if it "just works". If there are any major blockers that come up, I might not go down that path, but from the quick test I was able to whip up a year ago, it seemed feasible.
Post Reply