Moho Animation Forum

Hi, I've expended several days looking for an appropriate (or rather, easy/simple) way to start some kind of code version controlling during AS scripts development but (surprisely ) it's becoming a nightmare more than a solution and at this point I feel like almost every try to get used to proposed workflows and its concept is in over my head (Git's Submodules & Superprojects, Hg's Subrepositories, Symlink based dependencies...) cause I don't think I'm reaching any satisfactory conclusion...

I started trying to learn the basics about how Git (and, more recently, Mercurial (Hg)) works, but as soon as I try to think the way to manage shared files from different locations (like the ones intended to be stored in the "utilities" folder) between repositories in order to establish a consistent way of work, or simply try to adapt the repositories file structure system (which basically requires to have every involved file inside a single folder or "working directory") to the way AS folders structure imposes on how files must to be located, well... lets say everything start becoming (even) more and more complicated than it already is, and I think I start to feel the temptation to stop researching about all this and simply get back to my old (and horrible, BTW) way of work, which basically consisted of go making simple copies or backup of files with a little description about changes in their name from time to time. But of course I've never felt happy with this primitive way of work and I still want to move to another more beneficial solution, which at least provide some kind of reliable Version History and source code backup control, so although I how this is like a very "generic" issue beyond AS scripting field, I've finally decided to ask for advice here just in case someone has gone through a similar situation before and has something to say to the respect, or... about other possible alternatives or simpler workarounds or whatever, taking into account any AS scripting project will ever be so complex than what any Git/Mercurial kind of solutions are designed to manage, I mean, maybe I'm simply over-complicating things for what I really need, and I simply can't see the forest for the trees... After all, and as I said, I don't really need anything beyond Version History and appropriate code backup system, I mean, I can't even think on social coding and stuff like that for now, but I'd really like to start doing things *rightly* once and for all, and something tells me this matter could be like a very good starting point for it.

Hmmm, I hope I've been able to express myself around the issue at least but, summarizing, my main problems/concerns for what I've seen until now, are related to how shared files can be tracked by different projects (for example, files like: rl_utilities.lua) in order to avoid history messes/conflicts and the necessity of maintain them separately, plus... folders structure limitations (like how to correctly manage files outer the main repository folder). Well, greetings and thanks in advance for any help/correction/advice!

Hi again. Looking at Subversion (SVN) right now and, although everyone seems to agree this kind of Centralized Version Control Systems are not the way to go nowadays, at first sight I like some things I can see on it, like better support to external files (one of my main concerns) by that svn:externals (+) property or the more consolidated GUIs like TortoiseSVN. Not to mention that, even getting (more or less) the idea of why Distributed Version Control Systems are far superior under the majority of cases, I'm sure Centralized ones can be more "within my possibilities" right now so, although I'd like can follow the mainstream for once..., maybe I should get DVCS out of my head (at least for the moment) even if it's only for metal health...

Of course, I'm not sure about it yet, because with Subversion I'd have to renounce to some things I liked a lot about Git or Mercurial, like all the available free repositories (very specially Bitbucket) and without forgetting mainstream, but well... these are some of my thoughts right now just in case someone has something to say to the respect, either it be about personal experiences on similar scenarios (scripting or plugins development) or simply anything in favor of Subversion taking into account everything said.

Hi Ramon!
welcome to the complex world of coding!

I would like to give you a hand on this so here I go:

I deduce that there are some files that are common between different projects, right? What I don't understand is why that represent a problem?

Although you have different projects (or lets call them features) at the end, all they will be merged together, right? I mean, those features will work all together and won't produce different set of folder/files, Am I correct?

In git you work with a branch for each feature and with a master branch (which is the one which will collect all them).
The usual way to work is that you code one feature at a time, so you can perfectly work with a repository and different branches.

If you checkout one different branch, all the files associated to that branch are changed automatically and so you can make your tests on Anime Studio without troubles.

If some of those files are common for all the features and its change affects the other projects what you can do is this:
Guess that rl_utilities.lua is the file that is used by different features. Then guess you have two features being developed in two different branches:
rl_lost_as_tools (branch 1)
rl_smart_bone_helpers (branch 2)

What I would do would be this:
I would work with other branch called 'development' which will hold of the merge of the development features.
Say you are working on a modification on rl_lost_as_tools and you release the feature (that is you consider that it works fine and will be merged to the development one. Then you checkout 'development' and merge from 'rl_lost_as_tools'. Now 'development' has a modified version of rl_utilities.lua that includes its features.

Since it is a common file used everywhere you should update your other features with the recent change. So you have to checkout 'rl_smart_bone_helpers' and merge 'development' into it. Then all the common files that has been modified are updated and you'll see immediately if there are conflicts between 'development' and 'rl_smart_bone_helpers'. The sooner you merge and raise conflicts the easier to fix the conflicts will be.

If you develop both branches a lot, without merge together there will be very difficult merge problems to fix at the end.

You can use any helping feature from git or from other VSC but at the end you need to merge the changes all together. The sooner the changes are merged, the better.

Finally, once the development branch is fine, you release it to the master branch by merging it into the master.

I strongly recommend to use SourceTree from Atlassian. It makes git user's life easier
http://www.atlassian.com/software/sourcetree/overview

Good luck!
-G

Btw, I've sent you an email (gmail) to see if we chat on-line and can find a good solution for this.
-G

Cool! I was afraid this easily become into one of those "soliloquy" threads, so thank you first of all

Genete wrote:welcome to the complex world of coding!

Yeah, it has taken to me more than a year since your (wise) advice about it, but finally (and fortunately) I've been able to decide to face up the issue and, although in a very primary way for sure, I think I'm already tasting some of the sweet flavors of this new level of "control"

Genete wrote:I deduce that there are some files that are common between different projects, right? What I don't understand is why that represent a problem?

Although you have different projects (or lets call them features) at the end, all they will be merged together, right? I mean, those features will work all together and won't produce different set of folder/files, Am I correct?

Well, I never would have said either that so simple (and I thought common) situations, in which dependent or shared resources between projects (or rather, different AS tools) comes into play, could represent a problem for Version Control but, although it seems "possible" by means of the aforementioned solutions (Submodules/Subrepositories), it can complicate and limit workflows in a manner that I don't think I can deal with, for now at least... But, fortunately, involved elements are in my case very few, as said, basically the typical and unique "rl_utilities.lua" file which, being by its own nature intended/conceived to interact with ANY other independent project/tool's code, will be "susceptible" to be modified at ANY time and from WHICHEVER project/tool (and, therefore, different repositories) be in development process, and well, I'm simply not sure yet how this is managed by CVSs and if it could cause conflicts or history problems in the future (but I admit there are still a lot of concepts about CV's internal functioning that I totally ignore). OTOH, for your words I deduce it couldn't be a problem at all as long as I take special care about updates (which, although maybe kind of uncomfortable, is logic and convenient cause, after all and as you suggest, the idea behind this kind of shared resources is they must be compatible and "up-to-date" to every involved project (IOW, I want the last version of "rl_utiulities.lua" works fine with everything else: older AND new). So, assuming Git/Mercurial doesn't provide a solution equivalent to the named "svn:externals" (which, if I understood it well, should manage and take care about that kind of updates when needed), I think I'll have this main choices in order of preference...

1. Work with ONLY ONE "rl_utilities.lua" kind of shared files and SEPARATELY, that is; in a totally different/independent (and, therefore, external) repository, be making changes as needed and (and here is what I mainly wanted to avoid) be manually Copy/Pasting the "rl_utilities.lua" file into all the other involved repositories as needed, then, update them all. That way, if I'm not missing something (that I doubt), every repository will save its own snap of the file in history and I should be able to go dealing with "rl_utilities.lua" development and bug fixings by means of its own repository without major conflicts or history problems on dependent repos. Hmmm, I'm not totally sure but this sceario could be near to the solution you propose, isn't? Anyway, at least it seems to have some sense in my head... As I said, the main handicaps I see on this are (assuming it doesn't break any basic VCS rule) the manual managing and copy/pasting, although OTOH, this method will allow me apply the same "symlink" solution I'm using for normal projects/tools in order to solve the structure folders problems I mentioned, as I'll explain later.
2. [DISMISSED]I had in mind an alternative way consisting on go working with SEVERAL and different "rl_utilities.lua" copies inside each involved project/tool/repository, but I see now why this wouldn't be reliable at all, not only because it'd easily become a mess but it would complicate the "symlink" solution I was talking about, I mean, I should be updating their "symlink" in the AS "utilities" folder to target the WIP "rl_utilities.lua" copy each time I go making changes on them from the different projects/repositories, in order to can make ongoing AS tests.[/DISMISSED]

Well, I have to learn some more things about workflows in general and branching in particular, in order to can really say and feel comfortable with all the concepts, but I think I get the basics behind the workflow you propose and, precisely, the possibility to work in that way what made me discard the Subversion option in favor to a Distributed Control Version System like Git or Mercurial, concretely the last one, with which I star to feel happy as far as I've tested (not as much as I'd like ). But, if I understand it well, you are proposing something like a "super-project", or rather: "SUPER-REPOSITORY", which should contain ANY other project as long as any shared file between them exists, isn't? (otherwise, files interaction between totally independent repositories wouldn't be possible in Git or Mercurial as far as I know). Well, certainly that possibility have passed through my mind at some point but the lack of sureness around all this, made me start experimenting with independent "per project/tool" repositories , but this is something about I definitely want to delve and requires much more thinking for my part, as I like to start on the right foot if possible, cause I'm always afraid about if once initiated the workflow of choice, there will be a point of "no return" for deep changes in the way it was conceived.

About the VCS of choice, it was not an easy task... but I finally decided to start working with Mercurial as long as it seemed to me a little more "attainable" in concepts and I perceived the "TortoiseHg" option as a little more consolidated than the "SourceTree" one (which I just discovered the other day by a very opportune genbetadev article, BTW), but luckily (and under my little understanding) the Mercurial and Git main concepts and workflows can be very similar so I hope such decision doesn't affect too much in the way we could understand each other on this matters (?).

Well, I wouldn't stop.... but I think I'm still running my mouth too much and maybe I should stop here and leave other possible points to be discussed in the way you suggest . Again, thank you very much for be there trying to make this kind of "out of this world" things seem a little more bearable for dummies!


EDIT: Oh, about the "Symlinks" part... well, as I implied, I'm simply making use of relative Soft (there seems no need to go Hard) Symbolic Links in order to can separate working directories (repositories) from the imposed AS's folder structure in which the lua files must be placed in order to can make the typical (and so recurrent) "Alt Gr+F5" instant tests. And it seems to work like a charm and, as far as I can say, with no drawbacks! Cause this way I'm not really messing Git or Mercurial (which doesn't seem to support Symlinks very well, at least in the expected way) with them, but simply "cheating" AS about the real placement of the files. This can seem a silly/basic thing, but it solved at a stroke one of my main concerns and doubts around the entire issue.

By your (long) post I deduce that you want to work with different versions of AS and so, is that right?

Can you please explain in a short way the scenario of what you're dealing with? Number of files, its folder structure, its dependence and where are they going to be used.

Sometimes it is better to use the old quote: "divide and conquer" so keep that in mind if you want to give support for more than one AS version (remember that the API between versions has changed so it complicates more the things)


-G

Well, I'd like have made more progresses in the last two days, but here is how I started to organize my workspace:



As you can see, this scenario still matches with what I described in the first point of my previous post's list, that is, a "per tool" independent repository as well as independent repositories "per each shared file" (rl_utilities.lua in this case, although in the capture it doesn't appears initialized yet), basically... everything would be equally treated/managed under this scenario, and it's that simple! Sorry for not have provided this kind of "visual support" before... Anyway, my concerns about if this is the best way to manage the "lr_utilities.lua" dependancies still persist, but maybe I should stop worrying so much and start testing?

Anyway I'm still intrigued about the other option I thought you could be referring (what I branded as "SUPER-REPOSITORY" in my previous post) and, if possible, I'd like to clarify it just to ensure I'm not dismissing what maybe could be a potential solution to the shared files management issue, lets see if I can formulate it in a concise way: Taking the attached image as a reference, in which as you can see I'm simply creating independent/separated repositories under a normal folder called "dev", do you think that could be *advantageous* simply initialize the repository on "dev" folder (forcing this way everything inside "dev" folder by tracked and managed by means of such only "SUPER-REPOSITORY") instead go making individual "per project" repositories as I'm doing right now? It has some sense to me as long as projects have something in common (or rather, something to share) but I have not seen or read about anything similar to that possible solution and I've my reservations, not to mention that this way of work seems like a little more "dangerous", more like... if one be risking everything on one venture? OTOH, it'd totally contrast with you last "divide and conquer" advice, but even though I wanted to be totally sure about what you really think.

By your (long ) post I deduce that you want to work with different versions of AS and so, is that right?

Nope, as you can see, everything is much more simple than it could seem for my words... really, I didn't even think about several AS versions management! And now that you mention it... erm, I think I don't want even have to think about such thing for now (although, shouldn't "Clones" help on that?). Anyway, and fortunately, there are ways to avoid most of the possible errors caused by some API variations, but as I said, my concerns are still on the other aspects for now.


Well, thank you again! And I sincerely hope everything makes more sense this time...

Hi Ramon!
I'm sorry but you way of work will lead to problem. I'll try to explain it:

Say that you're working with lost_layer_tool. For this feature you need to modify two files:
rl_lost_layer_tool.lua
rl_utilities.lua

(Remember that you have two repositories one per file)

Say that you're working on it and you do three modifications (commits) to rl_lost_layer_tool.lua and two commits to rl_utilities.lua useful for that lost_layer_tool.

Say you haven't finished the feature completely and want to continue working with other feature like for example the smart_buttons. When I say you want to work with other feature I mean that it is possible that someone else can work in parallel with you.

So then you come back to smart_buttons and then notice that rl_utilities.lua is not compatible with smart_buttons due to the changes made by lost_layer_tool. What can you do? You can start a new branch at the rl_utilities repository in an earlier commit where it was compatible and do the modifications there.

Say you come back again to lost_layer_tool and logically you get back to the proper branch on the rl_utilities repository and to the modifications there.

Did you notice that there are two versions of the same file in the rl_utilities repository? So conclusion, each repository can have multiple version of each file (one in each branch). Did you notice that at the end, you need to merge rl_utilities branches in one master one to make it compatible with all the features? So if at the end, you'll need to maintain different branches and merge them into a single one, why keep all those repositories when you can do everything in one single one?

Conclusion, you only need one repository that should be on top of the higher folder where the files are. In our case it is at:
D:\Anime Studio Pro\scripts
because you need to control the versions of files at:
scripts\tool
scripts\ScriptResources
scripts\utility

Simply add all the files inside those folders to the version control system and start to work using a branch for each feature.

The logical way to work is explained here (it is for git but valid for mercurial too):
http://atlassian.com/git/workflows

-G

Genete! That's EXACTLY the kind of help/advice I was originally looking for (sorry I complicated/delayed it so much beating about the bush), so thank you very much

I mean, although work with only one "SUPER-Repository/Project" is something that had passed through my mind as the way to easily have every involved file (specially shared ones) under control by the VCS, I really didn't know if it could really have any sense from a learned person's POV or if it could entail unexpected drawbacks or whatever, plus it sounded to me like a potentially radical or difficult to manage solution... But, after your explanation I think I'm able to "glimpse" why definitely the benefits can be superior to the possible disadvantages I can think and it totally encourages me to try to change my mind accordingly this approach. Nevertheless... there are still some points on it about I'm not sure and which surely will require more thinking for my part, for example: What about if then, in "Bitbucket" or whichever, I want some projects/tools marked as private and some others as public? As I'm working with only one repository... must I assume I'd only have one choice for all of them? And only one Bug-tracker as well?? I could be wrong because I've not studied that parts in deep yet but, if that were the case, it sounds a little cumbersome to me and far to the idea I had in mind about how get things organized there...

Well, anyway, I'm definitely for start experimenting on that line (although fear to unknown incredibly slows down everything) and I hope can find/propose a more appropriate environment more soon than later, taking into account all the new & invaluable help. Well, greetings!


PS: Oh, I failed to mention that another important advantage about the working directory/environment you propose, is that (as long as the default AS's "scripts" directory by used as the repository itself), there would be no need to resort to Symlinks anymore, and (although for some reason, at first, I liked the idea of can maintain repositories independent to the default/official AS folders) that could be like another interesting point in favor of carefreeness and simplification.

I want some projects/tools marked as private and some others as public?

Just don't push your private branches to any public repository. That's all.
In case you need to upload the branches to a remote repository to work on different computers without worry to carry on the repository in a flash usb, then you probably should pay for the repository host. In github for example, all the public/open source repositories are free but you can have your private ones if you pay it.
-G