Retaining Backwards Compatibility in a Changing World

[LCfraser]

And frankly, I'd love to see some type hinting feature that we could use; it would be nice to be able to refer to a piece of data as something other than a string. Totally optional, but if you opt into it, then you can rely on it (i.e. violations would trigger on explictVariables being true). Might speed up the parser too, if it can compile something down to a type. Yes, it goes against LC's typeless system, which is why it should be optional, but man, coming from a traditional programming view and not being able to rely on my array of ints staying ints always leaves me faintly worried.

Would "watch this space" be a suitably mysterious comment to add here?

[malte]

Would "yay!!!" be an appropriate answer to the retoric question?

[jacque]

I'm sort of hoping that someone will come up with a good argument *for* maintaining backward compatibility, because I can't think of one.

Several thousand lines of code across hundreds of stacks that require a bug fix only available in a newer version. My situation.

My code is mostly shared in a main stack script, but if the bug is in any of the document stacks I'll need to update them all.

[mwieder]

Several thousand lines of code across hundreds of stacks that require a bug fix only available in a newer version. My situation.

Sure - but then you add one line of code to a mainstack that says "set the legacymode to true" or the engine automatically sees that the stack file version is older than <cutOffDate> and forces it true.

[jacque]

Several thousand lines of code across hundreds of stacks that require a bug fix only available in a newer version. My situation.

Sure - but then you add one line of code to a mainstack that says "set the legacymode to true" or the engine automatically sees that the stack file version is older than <cutOffDate> and forces it true.

Okay, I'm good with that. The stack file version would be new, if I'm compiling to fix a bug, but a flag property would work. If that can be implemented then I'm with the group that says "forge ahead."

[dave.kilroy]

Yes move forwards but with improved documentation tagging and search (so that we can filter for what applies to specific versions) - also consider implementing new error handling first so this can be used while updating legacy scripts.

error handling unification: Yep - this is a biggy. The refactoring makes it a much easier project to start looking at undertaking, but there is still a fair bit of engine legwork to do.

[AxWald]

Hi,

even if I'm by far not competent enough to judge in this case, I'd like to post a suggestion. To understand it, isn't it that we already have 2 lines of development? The "classic" 6.* line and the new "unicode & more" 7+ line?

Why don't we just fork LC? Idea:

1. 6.* is kept as-it-is as a common denominator, is not developed further much, but is still optimized and accelerated as a by-product of the new engine efforts. With the promise to keep it up, compatible and going - even if at a low development rate.
   .
2. Whereas 7.* is where the progress comes in. Where changes occur, where old sins get buried, and where a new LC emerges.
   .
3. As I understand a lot is in the pipeline, but even now LC 7.* still has such many backdrops that I wouldn't try it in my sensitive environments. I might be hardcore, but the footprint of LC 7.* compared to LC 6.* is so much bigger, my current habit of having countless LC-StandAlones fired up each hour might lead to problems with the 7.* versions.

So I'd wish to keep a 6.* version, without being completely disconnected from the development. I very well realize that the 7.* is developing rapidly, but still there's much more RAM & processor time required.

I'm all for further development, but this shouldn't kill previous efforts. We should keep a 6.*, for those that need it and are happy with the features. IMHO.

Have fun!

[Mikey]

The reason for not forking LC...yet, is because most of the LC community is made up of people that either never bothered to learn C and it's offspring, or want to forget they ever did. What I think Mark is thinking, or at least I hope he is thinking, is once we're in the LC8, i.e. LC Builder era, forking becomes much, much easier for this community. Obviously, I don't know what Mark or Kevin are thinking, but it would seem like a fairly straightforward effort. LC is open source, after all.

[makeshyft]

I think the engine should be improved and evolved, and you can only do so much for backwards compatibility. I am willing to go through upgrading my code if I want to switch engines.

My question is about one of your lines. I always use "the result" after I call a custom command and want to get its return value right after it runs.

MyCommand
put the result into result_var

how will this change effect this behavior? cuz it sounds to me like its totally not gonna work if the variable isn't global but local..unless its local to the script calling the command.

Thanks

Tom

[LCMark]

@jacque: The 'compatibility flag' idea would work perfectly for you in this case. Hopefully once we've compiled a more comprehensive list of potential changes we can more easily reason about the best way to implement the flag (is it a 'set' of options, or is it a single mode, or is it like the 'scriptVersion' idea which increments over time).

@AxWald: We are essentially maintaining two forks of the engine internally at the moment - 6.x and 7.x - and we've got reasonably good over the last year or so at the process of choosing where to fix things, and back-porting or forward-porting the changes. Ideally though, the effort for maintenance would be put into optimising 7.x so that there wasn't any question in people's minds about which to use. If you do have some code which is easily extractable and shows larger memory / processor usage then we'd be more than happy to take a look at it to help us improve the 7.x architecture - just drop an email to quality@livecode.com, or file a bug report.

@Mikey: 'forking' in the Open-Source world comes with a problem of licensing... You are more than welcome to fork the source and build your own engine - but its license will only be GPL. You cannot fork the engine source and then build it under commercial terms as the source-code you can download from GitHub is only conveyed under the GPL. Now, that being said, it might be that at some point it would make sense to put 6.x in 'community maintenance mode' - once we have the systems to allow automated building in place, we could provide a small amount of engineer oversight but rely on the community to patch 6.x. Under this remit we would probably only allow bugfix patches and not feature patches - as we don't really want to see 6.x and 7.x diverge in functionality. That being said though, the effort put into maintaining 6.x is probably far better put into ensuring the 7.x is more performant as it is where all the new goodies will be.

@makeshyft: I perhaps did not explain that change too well. Basically 'the result' would be directly tied to the return value of the last executed command in the script within a handler. i.e. When a handler returns a value, it will 'set the local result variable in caller', rather than 'set the global variable result'.

[benr_mc]

Add me to the omelette lovers (per Monte, i.e. prepared to break some eggs for a more satisfying result).

I would also note that these are not all of the same class. Some of Marks list I at least would be prepared to view as always wrong, now righted (eg arrays really should throw an error when attempting to convert to a string); others might be possible to improve without breaking compatibility, by using a new name-space (e.g. the naming of revXML and revDB functions reflects history of how first implemented - maybe this is the time to unify them into the general style of the language, with new semantics).

My biggest plea though would be to favour changes which can be made to break catastrophically.

I'd much rather open an old stack in a new version and have it throw a lot of errors, which I can work through one by one (and where it can't throw them at compile time, at least throw errors as functions are executed); than have it appear to work but undergo subtle behaviour changes which it will take a lot of time to trace back to something that silently did something different half an hour before. The changes - the omelette - is certainly worth paying something for; but the price we have to pay can be vastly reduced the more it can be obvious when code needs to be modified.

Ben

[dave.kilroy]

Yup - not only omelettes but:

My biggest plea though would be to favour changes which can be made to break catastrophically.

Along with new error/exception system to help us put everything back together again

[rjb]

I like Django's system: a clear roadmap of features that will be affected in the future. Right now, I think Django is at version 1.7 (and change). Version 1.8 will in fact remove features. This was announced back when 1.6 was released. Those features work just fine in 1.7, but they issue warnings. Developers have had about 2 years to get ready for their impending departure; since version 1.8 will no longer work if you use those deprecated features.

Identifying things as obsolescent way ahead of time sounds quite reasonable. I'd vote for that. People who want to implement some new feature brought by new version must do some coding anyway, so they may as well deal with deprecated features. However, this should be supplemented with supporting at least certain versions for longer. Support in terms of bug fixing I mean, so fewer people are forced to move to a major new release only because of bug fixes. May be LC could introduce the concept of long-term-support (LTS) and declare some branches as such (6.7 sounds like a candidate).

[pink]

A foolish consistency is the hobgoblin of little minds. (and I'll be honest, I didn't even realize that result was a global...)

I ran into one issue with a stack that would not function correctly when I moved from 6.7 to 7, and for that app I just keep using the older version. Everything new I've made in 7. If anything comes along that really requires I bring the old app into 7, I will address it. Otherwise, I'm content having multiple incarnations of Livecode on my hard drive.

I'm also willing to wager that some bright soul will develop a plugin to help search through scripts and find lines of code potentially affected by changes that are made to the syntax.

But since there hasn't really been any voice of dissent, I figured I would tell everyone a little story of something that is happening right now.

For the last 15 years, the company I work for has been using CFMC, which is scripting language and engine that manages CATI (computer assisted telephone interviewing) operations in our call center. It's an extremely expensive system that we have to subscribe to. At any time, there may be as many as 5 or 6 versions that are active, and every once in a while, a version gets put out to pasture.

Last September this happened, and the new version we needed to upgrade to required shelling out money for a new linux server. (Which made me happy, but not the people who had to pay for it).

Every fall we have a large project for a client who provides the scripted program to us (i.e. luckily it wasn't I who had to debug this beast). This project has been going for 9 years without incident with the same program (a few tweaks here and there but still mostly the same). But in September, the version of CFMC that we upgraded to completely broke the program. The programmers spent weeks on the phone with support working on bugs many of which are still not fixed. We spend a lot of time looking through data with fingers crossed that something horrible isn't happening behind the scenes. On my company's side of things, I've put in an excess of three times the usual number of hours I normal would have for this job. Production is down across the board. The main result is, our client is dumping CFMC for another language, and now so are we.

They had been considering the change for a while, but now they are determined. So who's to say there aren't developers who are on the fence about Livecode, and these changes could push them over.

On the other side... the result of this story for me is that I'm trying to see if I can build my own CATI system in Livecode

EDIT: was informed that several other companies that are working on this project are also dropping CFMC as their platform. So, one bad program and CFMC has lost 4 clients.

[mwieder]

Greg-

Thanks for the story from the trenches. Yeah, this sort of thing can happen. Sounds like it was pretty extreme in your case. And that can happen even without something like breaking compatibility, as you've seen with the 6.7 -> 7.x migration.

I should say, though, that the some of the things that we're holding onto because of backward compatibility are exactly the things that can be holding others off from adopting LiveCode. So as long as things don't break (the compatibility flag works, the syntax doesn't fall on its face, etc) we should be ok. I would hope that the compiler would be able to catch the deprecated syntaxes and flag them so they can be updated at compile time rather than runtime.