Lazarus

Announcements => Free Pascal => Topic started by: Martin_fr on June 22, 2021, 07:51:24 pm

Title: FPC & Lazarus moving to gitlab
Post by: Martin_fr on June 22, 2021, 07:51:24 pm
Hello All,

The Free Pascal and Lazarus teams are in the process of switching to Gitlab to manage their source code and issue reports.

In order to lower maintenance of their own infrastructure, a hosted solution
has been chosen, and an open source license was granted to the teams.

You can see the current progress at:

https://gitlab.com/freepascal.org

2 subgroups have been made:

https://gitlab.com/freepascal.org/fpc
https://gitlab.com/freepascal.org/lazarus-ide

Several testconversions of the SVN history to a Git repository have been
done, with ever improving results. All repositories will be switched to git.

A program has been written to convert Mantis bug reports to Gitlab issues;
it is very complete, and all available info will be converted.
You can see (partial) conversion results on the above projects.

The date for the final conversion has been established as the weekend of
17/18 july. People that wish to report bugs after that will have to create a
gitlab account in order to do so. (Those with a github account can normally
also use that account to log in with gitlab, see the gitlab login page.)

The conversion program will attempt to convert existing bugs using the
account of the new gitlab user. To map your gitlab user name (or ID) to the
mantis user, we ask that you file an issue in the mantis project of the
current bugtracker, assign it the category gitlab, and set the summary of
the bugreport to your gitlab account name or ID number (not both!).
https://bugs.freepascal.org/bug_report_page.php?project_id=4

All accounts that can be collected in this manner by 17 july will be used in the final conversion.

All necessary information to connect to gitlab will be collected in the FPC &
Lazarus Wiki. Several pages have already been set up:

https://wiki.freepascal.org/FPC_git
https://wiki.freepascal.org/FPC_git_concepts
https://wiki.freepascal.org/SVN_to_GIT_Cheatsheet

These pages will be updated with the correct URLS when the final conversion happens. The FPC & Lazarus websites will also be adapted with new instructions.

For the FPC & Lazarus teams,
Martin (Message originally by Michael)
Title: Re: FPC & Lazarus moving to gitlab
Post by: Fred vS on June 22, 2021, 08:11:24 pm
  :)   ;)   ;D

Fre;D
Title: Re: FPC & Lazarus moving to gitlab
Post by: MarkMLl on June 22, 2021, 08:32:01 pm
Does this mean that it will no longer be possible to refer to a particular stage of trunk development as "revision such-and-such", and to step forwards and backwards from that point using a contiguous sequence of revision numbers?

MarkMLl
Title: Re: FPC & Lazarus moving to gitlab
Post by: Fred vS on June 22, 2021, 08:38:04 pm
Does this mean that it will no longer be possible to refer to a particular stage of trunk development as "revision such-and-such", and to step forwards and backwards from that point using a contiguous sequence of revision numbers?

MarkMLl

This? (https://stackoverflow.com/questions/4120001/what-is-the-git-equivalent-for-revision-number)
Title: Re: FPC & Lazarus moving to gitlab
Post by: dr2e on June 22, 2021, 08:41:13 pm
The Free Pascal and Lazarus teams are in the process of switching to Gitlab to manage their source code and issue reports.
10MB ought to be enough for anybody (http://gitlab.com/gitlab-org/gitlab/-/issues/20061), yeah.
Title: Re: FPC & Lazarus moving to gitlab
Post by: MarkMLl on June 22, 2021, 08:59:36 pm
Does this mean that it will no longer be possible to refer to a particular stage of trunk development as "revision such-and-such", and to step forwards and backwards from that point using a contiguous sequence of revision numbers?

MarkMLl

This? (https://stackoverflow.com/questions/4120001/what-is-the-git-equivalent-for-revision-number)

Thank you Fred, I presume you mean that the sequential revision numbers will no longer exist but I would appreciate confirmation.

MarkMLl
Title: Re: FPC & Lazarus moving to gitlab
Post by: GAN on June 22, 2021, 09:40:25 pm
Excellent news (and decision). Thank you.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on June 22, 2021, 09:43:48 pm
Does this mean that it will no longer be possible to refer to a particular stage of trunk development as "revision such-and-such", and to step forwards and backwards from that point using a contiguous sequence of revision numbers?
Mainly, yes.

The exact details on the below are still being evaluated. Outcome not yet known....
There will be options with "git describe", and such give you a way to refer to a commit in a way that tells you N commits after named known point.


If you do have a commit, you can walk to its parents easily (but not its children)
SOME_HASH_OR_NAME~1

the number gives how many parents up.


Title: Re: FPC & Lazarus moving to gitlab
Post by: MarkMLl on June 22, 2021, 10:26:49 pm
Does this mean that it will no longer be possible to refer to a particular stage of trunk development as "revision such-and-such", and to step forwards and backwards from that point using a contiguous sequence of revision numbers?
Mainly, yes.

The exact details on the below are still being evaluated. Outcome not yet known....
There will be options with "git describe", and such give you a way to refer to a commit in a way that tells you N commits after named known point.


If you do have a commit, you can walk to its parents easily (but not its children)
SOME_HASH_OR_NAME~1

the number gives how many parents up.

In that case I'm abandoning trunk. Git is indisputably an adequate tool for collaborative work, it's not suitable for software distribution.

MarkMLl
Title: Re: FPC & Lazarus moving to gitlab
Post by: dbannon on June 23, 2021, 03:23:44 am
In that case I'm abandoning trunk. Git is indisputably an adequate tool for collaborative work, it's not suitable for software distribution.

see https://github.com/graemeg/lazarus

I have been getting Lazarus Trunk from github for the last year, very easy to step back one or many 'revisions', you work with the github web page open, you can see all the IDs, scroll down and see the diffs.  I imagine gitlb will be able to do the same.

You'll love it !

Davo
Title: Re: FPC & Lazarus moving to gitlab
Post by: berocoder on June 23, 2021, 06:48:56 am
I use Github myself and assume Gitlab is similar. Very good decision!
Title: Re: FPC & Lazarus moving to gitlab
Post by: PascalDragon on June 23, 2021, 09:02:25 am
The Free Pascal and Lazarus teams are in the process of switching to Gitlab to manage their source code and issue reports.
10MB ought to be enough for anybody (http://gitlab.com/gitlab-org/gitlab/-/issues/20061), yeah.

The size restriction on Mantis is 2 MB, so I don't see a problem there. If a user would need to upload that size of source (as that is what we're dealing with) in addition that being compressed then they didn't minimize the problem accordingly and we'd request them to do that anyway.

Distribution of releases will continue from our own server anyway.

In that case I'm abandoning trunk. Git is indisputably an adequate tool for collaborative work, it's not suitable for software distribution.

I really don't see the issue you have there. You pick your commit that you know works and for yourself you can even create a local tag so that you know it's your working commit. Then you fetch new commits, switch to main, check whether that's working for you and tag anew. If not you switch back to your previous tag.
Title: Re: FPC & Lazarus moving to gitlab
Post by: JuhaManninen on June 23, 2021, 09:34:56 am
In that case I'm abandoning trunk. Git is indisputably an adequate tool for collaborative work, it's not suitable for software distribution.
There must be a misunderstanding from your side.
SVN was not used for software distribution either. It is for revision control and used by people who are interested in the project's development and/or participating in it.
Git is used for the same purpose.
Releases will be distributed as before.

A sequential revision number is gone, thus it will be useful to list date/time together with a hash ID.
Mentioning the hash ID, date/time, author and commit message of a revision gives enough information for any purpose.
Title: Re: FPC & Lazarus moving to gitlab
Post by: MarkMLl on June 23, 2021, 09:50:07 am
In that case I'm abandoning trunk. Git is indisputably an adequate tool for collaborative work, it's not suitable for software distribution.
There must be a misunderstanding from your side.
SVN was not used for software distribution either. It is for revision control and used by people who are interested in the project's development and/or participating in it.
Git is used for the same purpose.
Releases will be distributed as before.

A sequential revision number is gone, thus it will be useful to list date/time together with a hash ID.
Mentioning the hash ID, date/time, author and commit message of a revision gives enough information for any purpose.

In the case of trunk, svn was effectively being used for distribution. I've seen Git being spectacularly incapable of filling the same role, e.g. with Mycroft where the /only/ way of running it is to have a complete repo clone.

Statement of fact: I've got too much else on my plate to be prepared to continue with trunk since the change will involve learning a new toolset. Sorry.

MarkMLl
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on June 23, 2021, 10:55:47 am
In the case of trunk, svn was effectively being used for distribution. I've seen Git being spectacularly incapable of filling the same role, e.g. with Mycroft where the /only/ way of running it is to have a complete repo clone.

there is a shallow clone

git clone --depth=1  url folder

And after that, you get the complete update, so yes you keep history from that point forward. But the disk usage for that in amazingly effective/low.

- I have the entire history (just git data, no checked out files / .git folder) in 227 MB.
- On the other hand the .svn folder of my svn checkout is 918 MB.

I don't have the size of a flat git clone.

To be fair, I maintain the git folder, and have it re-packed occasional. Otherwise it may be about 350ish MB. Still better that svn.
The .svn folder is whatever svn did by itself.

** EDIT: so I did a "clean up" in my svn folder and it is now 197 MB. So if you do maintenance, you save a few bytes compared to git. But my git (ever after an unmaintained period) never got even close to half as big as the svn was when unmaintained.


There is also a zip download via gitlab.
You can select any revision and get it as zip.
Title: Re: FPC & Lazarus moving to gitlab
Post by: MarkMLl on June 23, 2021, 12:05:08 pm
In the case of trunk, svn was effectively being used for distribution. I've seen Git being spectacularly incapable of filling the same role, e.g. with Mycroft where the /only/ way of running it is to have a complete repo clone.

there is a shallow clone

git clone --depth=1  url folder

Yes, I know: been using it for years. And I've seen projects- I gave an example- where that doesn't work.

Unless there is some way that the requirement that --depth=1 always delivers a valid snapshot can be "baked into" a project's Git configuration then there's a risk that sooner or later somebody will screw it.

MarkMLl
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on June 23, 2021, 01:03:57 pm
Yes, I know: been using it for years. And I've seen projects- I gave an example- where that doesn't work.
Ok, I missed that, actually googled it. Looks that it was a config fault by the project, or I found something else?
In any case errors happen with software.
SVN is not better. It does screws up for me... It does so, regularly when I switch the fpc-build between fixes and trunk. I got a script now, that remove all files, and does a new checkout. (I guess most people do not have that issue, but I am the unlucky one). So for me, git will hopefully fix an existing problem.

But to conclude, every change that happens in the world will be positive for some, and less good for others. The hope is that the amount of those benefiting will grossly outweigh the amount of those who don't.

We believe that this (more benefits) will be the case here.
Title: Re: FPC & Lazarus moving to gitlab
Post by: El Salvador on June 23, 2021, 01:06:36 pm
So Mantis will be disappear (or will be lock for history)?
Title: Re: FPC & Lazarus moving to gitlab
Post by: marcov on June 23, 2021, 01:07:11 pm
So Mantis will be disappear (or will be lock for history)?

Disappear, its machine is end of life.
Title: Re: FPC & Lazarus moving to gitlab
Post by: marcov on June 23, 2021, 01:09:44 pm
- On the other hand the .svn folder of my svn checkout is 918 MB.

Should be 300 or so. Probably an old repo that you don't regular vacuum.  (tortoisesvn a tick in the cleanup command).
Title: Re: FPC & Lazarus moving to gitlab
Post by: MarkMLl on June 23, 2021, 01:35:46 pm
Yes, I know: been using it for years. And I've seen projects- I gave an example- where that doesn't work.
Ok, I missed that, actually googled it. Looks that it was a config fault by the project, or I found something else?
In any case errors happen with software.

The developers' attitude was that it was intentional, and that anybody getting the Python programs from the repository was expected to have the entire repo on their hardware. Since I was running it on a PC that was bearable (a "mere" 2.8Gb), I don't know whether anybody running it on target hardware that the project supplied or recommended (basically, either a Raspberry Pi or a system embedded in e.g. a car) would find themselves in the same situation.

I am not prepared to get into an argument with the FPC/Lazarus teams if they make a similar configuration choice, or screw things up and go defensive. I am still rankling over the breaking change to FileExists()... granted that it was documented, but I still feel it was handled very badly and doesn't reflect well on the project.

Quote
SVN is not better. It does screws up for me... It does so, regularly when I switch the fpc-build between fixes and trunk. I got a script now, that remove all files, and does a new checkout. (I guess most people do not have that issue, but I am the unlucky one). So for me, git will hopefully fix an existing problem.

But to conclude, every change that happens in the world will be positive for some, and less good for others. The hope is that the amount of those benefiting will grossly outweigh the amount of those who don't.

We believe that this (more benefits) will be the case here.

I respect your decision and am not trying to pressurise you to reverse it.

However from my POV I've spent a lot of the last 20 years tinkering with non-x86 based computers and even longer dealing with minority OSes, mostly because of company policy that they should be used because they were technically elegant and were more robust in the face of error or intrusion. While that has increased my skills in some areas it has meant that in others I'm somewhat "behind the curve", and I quite simply can't afford to spend time going off on any more tangents.

As somebody who is not a core developer and has no (immediate) need to use Git or a Git-oriented hosting service for anything other than getting a current (.zip) snapshot, continued involvement with Trunk is tangential to everything else that I'm trying to do. In particular, I am not prepared to get myself into a situation where some needed facility of Trunk suddenly turns out to be broken, where if I want it to be fixed I've got to triage the release history myself, and where I find I can't easily revert to an older version because the .lpi or .lfm format has changed and I need a working Trunk to massage it back to a compatible format.

MarkMLl
Title: Re: FPC & Lazarus moving to gitlab
Post by: PascalDragon on June 23, 2021, 01:49:53 pm
As somebody who is not a core developer and has no (immediate) need to use Git or a Git-oriented hosting service for anything other than getting a current (.zip) snapshot, continued involvement with Trunk is tangential to everything else that I'm trying to do. In particular, I am not prepared to get myself into a situation where some needed facility of Trunk suddenly turns out to be broken, where if I want it to be fixed I've got to triage the release history myself, and where I find I can't easily revert to an older version because the .lpi or .lfm format has changed and I need a working Trunk to massage it back to a compatible format.

You'd have to do the triaging with SVN as well cause you can't necessarily know what broke it. And bisecting such a revision is even more comfortable with Git, cause it provides commands for that (no more counting revisions by hand).
Title: Re: FPC & Lazarus moving to gitlab
Post by: marcov on June 23, 2021, 01:58:53 pm
You'd have to do the triaging with SVN as well cause you can't necessarily know what broke it. And bisecting such a revision is even more comfortable with Git, cause it provides commands for that (no more counting revisions by hand).

Yeah, but usually you take the weight of the revisions (e.g. makefile regenerations/minor edits or unrelated minor target changes) into consideration so that each side of the bisection has about equal "weight".
Title: Re: FPC & Lazarus moving to gitlab
Post by: PascalDragon on June 24, 2021, 08:59:00 am
You'd have to do the triaging with SVN as well cause you can't necessarily know what broke it. And bisecting such a revision is even more comfortable with Git, cause it provides commands for that (no more counting revisions by hand).

Yeah, but usually you take the weight of the revisions (e.g. makefile regenerations/minor edits or unrelated minor target changes) into consideration so that each side of the bisection has about equal "weight".

In all honesty: in those moments I've done a bisection with SVN (be it with FPC or at work), I've simply counted the number of revisions (or more precisely: used TortoiseSVN's "number of selected revisions"). Much faster, much easier than to "interpret" the contents of the commit.
Title: Re: FPC & Lazarus moving to gitlab
Post by: MarkMLl on June 24, 2021, 09:25:23 am
In all honesty: in those moments I've done a bisection with SVN (be it with FPC or at work), I've simply counted the number of revisions (or more precisely: used TortoiseSVN's "number of selected revisions"). Much faster, much easier than to "interpret" the contents of the commit.

Which as I've said, is probably the main thing that bothers me. I note that Martin's doing some interesting work on tags... and I note obviously the amount of functionality that Git exposes which can be used (and abused) by maintainers. But for the moment at least the change is enough to push me off Trunk.

MarkMLl
Title: Re: FPC & Lazarus moving to gitlab
Post by: marcov on June 24, 2021, 09:43:36 am
In all honesty: in those moments I've done a bisection with SVN (be it with FPC or at work), I've simply counted the number of revisions (or more precisely: used TortoiseSVN's "number of selected revisions"). Much faster, much easier than to "interpret" the contents of the commit.

Maybe, if you can give it a manually established, cleaned set. I mostly bisected fpdoc/fcl-passrc in the last few years. It probably also depends on how complex your test is that exposes the issue (and mine involved several minutes of chm building and manually testing with the windows chm help).

Extra unattended builds only warm up the earth, but extra tests can be painful.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on June 24, 2021, 01:02:31 pm
Yeah, but usually you take the weight of the revisions (e.g. makefile regenerations/minor edits or unrelated minor target changes) into consideration so that each side of the bisection has about equal "weight".

Well, I've gone a similar, but 3rd way...

If (and only if) I knew the file/folder that likely caused the issue (i.e, a bug in SynEdit, likely caused by a commit to a SynEdit file).

I displayed a filtered log, and then visually picked the middle.
Or even read commit messages, and ignored "updated translations"

However, this is  "visually picked the middle" because revisions are not continuous in that filtered log. (So the svn advantage is gone).

Of course, this only works if the filtered list, is small enough....

----

But, since the "svn advantage is gone", that can be done in GIT too. I can display the filtered log. I can see the list, and pick the middle.



On the other hand, and here comes the real difference....

I have not yet done that, but I will most likely in future.

I can write a script (or I might adapt lazbuild for that) => to return an exit code, indicating if all was fine (testcase run and succeeded) or not.

And then I can give that script to git. And  can go, get my coffee, and git will bisect the revison without me doing any further work.

And if git does it all on its own, I do not care if the bisect need a few searches more or less.

(Yes, I can write a wrapper for svn too. But git already includes it)

Title: Re: FPC & Lazarus moving to gitlab
Post by: devEric69 on June 24, 2021, 01:33:51 pm
Question ( minor, and maybe silly ) from a novice (I don't know much about Git on a desktop workstation, and nothing about Continuous Integration a.k.a. CI or Continuous Delivery a.k.a. CD ): will the same "tree" semantics be kept on GitLab (i.e. "trunk", "branches", tag on "fix_", ...) rather than "master", "develop", "mainstream", ...?
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on June 24, 2021, 01:59:21 pm
will the same "tree" semantics be kept on GitLab (i.e. "trunk", "branches", tag on "fix_", ...) rather than "master", "develop", "mainstream", ...?

Not actually yet decided.
Also this decision may happen per team.

The current status is, that afaik both projects will keep (at least for some time to come) a development branch, that acts like the current trunk. (the name of it may be different or not).

Equally for now, the fixes branches will be kept, and merging will be done from development to fixes (as it has been sofar / actually, I have for Lazarus SVN seen some bug fixes made to fixes, and then go to trunk.... but in either case trunk (by any name) still  is the development branch.)

Yet the above does not exclude that feature branches could be used. That has happened in svn too.
Title: Re: FPC & Lazarus moving to gitlab
Post by: marcov on June 24, 2021, 02:02:10 pm
However, this is  "visually picked the middle" because revisions are not continuous in that filtered log. (So the svn advantage is gone).

More or less that is what I did too. I didn't really use revision values since I was mostly only interested in changes in certain dirs. 

Quote
Of course, this only works if the filtered list, is small enough....

I already have code that contains revision lists in tlists in the mergelogs. In such case a "middlepoint" (tlist[count div 2]) would be trivial.

Quote
But, since the "svn advantage is gone", that can be done in GIT too. I can display the filtered log. I can
see the list, and pick the middle.

All my scripts (mostly FPC TProcess based) and every other infrastructure is invalidated. Worse, git seems to want bash for nearly everything which makes scripting a pain on Windows, specially when multiple driveletters are involved.

I'm reluctant to do this all again, in case the gitlab thing doesn't work out in a few years, and we go back to hosted.

Quote
I can write a script (or I might adapt lazbuild for that) => to return an exit code, indicating if all was fine (testcase run and succeeded) or not.

And then I can give that script to git. And  can go, get my coffee, and git will bisect the revison without me doing any further work.

If the work was worth writing a bullet proof automate script for in the first place. It rarely is. (I shudder at writing a test for CHM integrity)

Quote
(Yes, I can write a wrapper for svn too. But git already includes it)

Yes. But the move invalidates more scripting, than these kind of limited use gadgets actually add. Instead we have to do with some pre-cooked gitlab pages.

Also, getting scripting crossplatform will be harder. I used FPC for that, but with GIT in special bash shells that will be harder.
Title: Re: FPC & Lazarus moving to gitlab
Post by: marcov on June 24, 2021, 02:05:07 pm
"master"

master is deprecated nowadays, the new suggested name is "main".
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on June 24, 2021, 02:35:59 pm
All my scripts (mostly FPC TProcess based) and every other infrastructure is invalidated. Worse, git seems to want bash for nearly everything which makes scripting a pain on Windows, specially when multiple driveletters are involved.
What "shell" are you currently using?

git is callead as a command/exe.
I have used it from
cmd  => so very seldom, because I dislike cmd.
the shell that came with git-for-windows
mingw/msys

Quote
I'm reluctant to do this all again, in case the gitlab thing doesn't work out in a few years, and we go back to hosted.
This is nothing to do with gitlab, but with git.

If we moved svn to be hosted by sourceforge, it still be svn.

Our git repro remains valid, whoever hosts the server part. If gitlab becomes unavailable then other hosters, or self hosting are very possible (but right now the choice is gitlab)

Quote
Quote
(Yes, I can write a wrapper for svn too. But git already includes it)

Yes. But the move invalidates more scripting, than these kind of limited use gadgets actually add. Instead we have to do with some pre-cooked gitlab pages.

Also, getting scripting crossplatform will be harder. I used FPC for that, but with GIT in special bash shells that will be harder.
I don't quite follow, including I am not sure about what is current, and what is future.

Of course existing scripts (or fpc apps) that call svn, must be changed to call git, and they must be adapted to the new syntax, the  new output, and maybe the concept.
That is a once off for git.
As long as we stay on some git, that stays valid.

And where git is to call a script or program (like in bisect) that can be an fpc program. Or a script that first builds the fpc app, and the calls it.

sorry if the last part is off... As I said not sure about interpreting your statement...
Title: Re: FPC & Lazarus moving to gitlab
Post by: marcov on June 24, 2021, 03:05:41 pm
All my scripts (mostly FPC TProcess based) and every other infrastructure is invalidated. Worse, git seems to want bash for nearly everything which makes scripting a pain on Windows, specially when multiple driveletters are involved.
What "shell" are you currently using?

I use the default shell on most targets. On *nix this is usually some sh derivate, on Windows
the default shell, cmd.exe.   As said, my experiences with bash shells on Windows is bad (even though I'm a 26 years long sh/Bash user). You can't copy paths from the explorer any more (due to \ vs /), driveletters require special handing, quoting rules etc are different etc.

Worse, the whole msys movement has actually regressed the situation, in that there were more mingw binaries working with proper windows conventions ten years ago, than now.

Quote
git is callead as a command/exe.
I have used it from
cmd  => so very seldom, because I dislike cmd.
the shell that came with git-for-windows

And does git properly understand driveletters ?

Quote
This is nothing to do with gitlab, but with git.

If we moved svn to be hosted by sourceforge, it still be svn.

True. The example was bad. It was just a remark about not wanting to invest heavily again, since this might also be temporary again. For windows users doubly so, since the degree to which git+associated tools are ported to Windows might vary over time.

Quote
[Of course existing scripts (or fpc apps) that call svn, must be changed to call git, and they must be adapted to the new syntax, the  new output, and maybe the concept.

Yes. Every done investment is void.   The more complex ones are in the FPC release and fixes branch maintenance realm though, which will now be done by Florian. So I won't have to redo them.

Quote
That is a once off for git.

It was a one of for subversion too at one point :-)

Quote
re git is to call a script or program (like in bisect) that can be an fpc program. Or a script that first builds the fpc app, and the calls it.

If git passes things like drivelettes and paths consistently yes. Otherwise you have to hack around that again, to interpret /drive/c etc.

I've had some extremely bad experiences with this in the FPC early years with cygwin gdb.

The only sane solution is to minimize the use and so usually that means that things like compelex scripting are out

But at least in the GDB case there was no choice. Now we are leaving a perfectly windows integrated tool for one that is not.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on June 24, 2021, 03:55:13 pm
As said, my experiences with bash shells on Windows is bad (even though I'm a 26 years long sh/Bash user). You can't copy paths from the explorer any more (due to \ vs /), driveletters require special handing, quoting rules etc are different etc.
Quote
And does git properly understand driveletters ?

Well, if run from cmd then yes, as far as I tested.
If run from a unix like sh/bash/... then no.

First of all, git expects to be run inside the repro dir (can be in a subdir, but not outside). At least afaik.
So most of the time you switch into that dir first.

But the following 2 examples worked fine, run within cmd.

The first part points to an existing repro on my b hard-drive.
The 2nd specifies a target, which is a new folder beneath the current dir (current dir was on c drive)
Code: Bash  [Select][+][-]
  1.     git clone b:\GITLAB_TEMP_WORK\t2_gitlab_conv_laz  temp

And when I am inside the git dir, and want the log for a specific file (again, run from cmd)
with backshlash
Code: Bash  [Select][+][-]
  1.     git log -n3 --oneline ide\main.pp

So it seems to work, but I have no in depth testing/experience with it. cmd is not to my personal likings.

Quote
True. The example was bad. It was just a remark about not wanting to invest heavily again, since this might also be temporary again. For windows users doubly so, since the degree to which git+associated tools are ported to Windows might vary over time.
s/git/svn/  => and the statement is either as likely as, or even more worrying.

I think the fact that any software might cease to exist (entirely or partly) is not an argument (unless there are actual signs that it is about to happen.)

Here we could same as good say, Windows users may have a problem, because Windows may cease to exist (it already has a subsystem for linux, it may just drop the system around it, and be linux in itself...)  ;)

Sorry, but I think your statement called for a slightly overdone answer....

Quote
Yes. Every done investment is void.   The more complex ones are in the FPC release and fixes branch maintenance realm though, which will now be done by Florian. So I won't have to redo them.

Yes, that is the nature of change. It usually requires investment. And if it is done correctly, in the end the benefits will be more than the cost.

However, the distribution of benefits and cost is not always fair. Some loose out, some win. The aim is that the amount of winners is massively higher.

Leaving todays knowledge about the environment aside, would you rollback the invention and spread of the automobile? Because lots of the people who cared about horses (for horse drawn carriages) lost their job?


Quote
Quote
...Of course existing scripts (or fpc apps) that call svn, must be changed ...
That is a once off for git.
It was a one of for subversion too at one point :-)
So you agree?
I guess you talk about the CVS to SVN?
So did the change for fpc turn out to be good? Or would it be better, if fpc was still using cvs?

Quote
If git passes things like drivelettes and paths consistently yes. Otherwise you have to hack around that again, to interpret /drive/c etc.

I've had some extremely bad experiences with this in the FPC early years with cygwin gdb.

The only sane solution is to minimize the use and so usually that means that things like compelex scripting are out

But at least in the GDB case there was no choice. Now we are leaving a perfectly windows integrated tool for one that is not.
See initial part of my answer.
Title: Re: FPC & Lazarus moving to gitlab
Post by: marcov on June 24, 2021, 04:38:01 pm
Well, if run from cmd then yes, as far as I tested.
If run from a unix like sh/bash/... then no.

Well that is something.

Quote
First of all, git expects to be run inside the repro dir (can be in a subdir, but not outside). At least afaik.
So most of the time you switch into that dir first.

When I wrote it, I was mainly thinking about export, but also to reduce commands to single lines again, since git seems to often need multiple commands with options for simple operations.

Helper apps to embed the checkout hash into apps also matter.

Though maybe tortoise git will eliminate them.  And maybe I should lighten up on the scripting front since I won't have to recreate the heaviest ones.

Quote
Quote
True. The example was bad. It was just a remark about not wanting to invest heavily again, since this might also be temporary again. For windows users doubly so, since the degree to which git+associated tools are ported to Windows might vary over time.
s/git/svn/  => and the statement is either as likely as, or even more worrying.

No. SVN was well integrated into Windows from the start. It didn't come with bash, and all needed basic commands were single issue without the need for scripting. It also needed minimal configuration. (though still more than CVS, but that was because CVS had awkwardly weak authentication/encryption)

Quote
This is why
I think the fact that any software might cease to exist (entirely or partly) is not an argument (unless there are actual signs that it is about to happen.)

We'll see. I just got horrible vibes from everything GIT. The constant pushing of bash might have been something that triggered me.

Quote
Here we could same as good say, Windows users may have a problem, because Windows may cease to exist (it already has a subsystem for linux, it may just drop the system around it, and be linux in itself...)  ;)

Nobody knows anything what will happen on the real long term. Usually it is also never so black and white, technologies more often change markets and heavily mutate than outright disappear.

But yeah, the year of Linux on the desktop might finally happen, and Linux might break out of its current server and power user audience. But I won't hold my breath after hearing such things about Linux longer than that I use windows.

The naive user sees the name of the product survive and assumes continuity that is not really there in reality. This is the danger of superficial analogies like the car one that you produced..

That something survives doesn't mean users on all paths are safe. Zeppelins still exist, but aren't as dominant as some people would have hoped. And then I'm not even asking Hindenburg passengers for comments ;-)

All joking aside, just survival of a concept doesn't mean that details investments are safe. To stay in your car analogy, early movers in post combustion car are writing off investments in hybrid vehicles because in many markets all electric is moving much faster than expected.

 
Quote
Yes, that is the nature of change. It usually requires investment. And if it is done correctly, in the end the benefits will be more than the cost.

However, the distribution of benefits and cost is not always fair. Some loose out, some win. The aim is that the amount of winners is massively higher.

If a proper cost/benefit analysis was done in the first place. I'm not convinced this is actually the case.

Compiler devels wanted (easier?) forward and local branches, and the server was up for renewal which was tempting for the people maintaining the infrastructure. And that actually forced the issue, not the GIT "features".

All the rest is piled sentiment on to make it palatable.

Quote
Quote
Quote
...Of course existing scripts (or fpc apps) that call svn, must be changed ...
That is a once off for git.
It was a one of for subversion too at one point :-)
So you agree?
I guess you talk about the CVS to SVN?

Yes. But that was different, because subversion had advantages for all user groups, not just a happy few that heavily use forward branches.

Also CVS development actively point to SVN as a successor, it was leaving a phased out (EOL) product. This is not the case with svn->GIT, and the tools are in many case a regression with git.

Quote
So did the change for fpc turn out to be good? Or would it be better, if fpc was still using cvs?

Definitely positive, but it still cost the project over an year, and while a bigger change in feature set, the amount of features used from the VCS was actually less. Little scripting and fixed procedures (e.g. for release and fixes branch maintenance).

So I expect it to be worse in this case, with smaller gains (which are also very unevenly distributed, and for some it might remain a netto minus).

I was initially negative about CVS->SVN too, I considered the change not worth the advantage (which back then for the common user was mostly renaming, though also back then the compiler devels were only interested in branching).

But when the migration got closer and more detailed, I got more positive.  Many small things were better. Also the EOL status became more prominent in the period between initial talks about migration and the actual transition. And Peter (and Jonas too iirc) prepared it very well.

Neither of those cases are now the case. The windows tooling is a step back, the commands are more complicated, the preparation less. And most importantly, SVN was not EOL.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Kays on June 24, 2021, 04:55:04 pm
[…] The Free Pascal and Lazarus teams are in the process of switching to Gitlab to manage their source code and issue reports. […]
That’s great news! Kudos to all that are involved in the migration process, writing and testing migration scripts and, let’s not forget, dealing with the non-technical issues.

[…] Statement of fact: I've got too much else on my plate to be prepared to continue with trunk since the change will involve learning a new toolset. Sorry.
That’s fair. I mean, “we”, collectively as the FPC project, are (or should have been) aware that some contributors will stop contributing just because of that, while at the same time we ensure the project’s longevity by finally switching to more “modern” tools like Git.
[…] But to conclude, every change that happens in the world will be positive for some, and less good for others. The hope is that the amount of those benefiting will grossly outweigh the amount of those who don't. […]

"master"
master is deprecated nowadays, the new suggested name is "main".
It ain’t “deprecated”, you know that, right? It’s just that some people like to force their world view, their ideology on others. Case in point “master” supposedly implied owning slaves. It’s rather fascinating that 99% of the English speaking world, i.e. first and second language English speakers mostly living outside of the US, don’t even understand the fuss about “master” yet we still have to put up with that shit, that language imperialism.
Title: Re: FPC & Lazarus moving to gitlab
Post by: marcov on June 24, 2021, 05:02:06 pm
"master"
master is deprecated nowadays, the new suggested name is "main".
It ain’t “deprecated”, you know that, right? It’s just that some people like to force their world view, their ideology on others. Case in point “master” supposedly implied owning slaves. It’s rather fascinating that 99% of the English speaking world, i.e. first and second language English speakers mostly living outside of the US, don’t even understand the fuss about “master” yet we still have to put up with that shit, that language imperialism.

Yeah, that. But since we can now avoid exactly such discussions in the future from both at no real cost (since as new git repos we are not invested in "master"), it is sensible to do so, regardless of what you think about the actual change.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on June 24, 2021, 05:20:37 pm
but also to reduce commands to single lines again, since git seems to often need multiple commands with options for simple operations.
You can configure git aliases.
Code: Bash  [Select][+][-]
  1. git config alias.slog 'log --oneline'
  2. git slog -n 3

There are also aliases where git calls an external command, usually a shell. Not tested with anything else. Maybe it will do with cmd, but I do not know that.

Quote
Helper apps to embed the checkout hash into apps also matter.
Code: Bash  [Select][+][-]
  1. git describe
However it requires the correct tags to be set.

Quote
No worries, I returned the favour. Don't wrestle with pigs. You get dirty, and the pig likes it :-)
Well, I might like it too. And I love bacon. :) 

Quote

Quote
I guess you talk about the CVS to SVN?
Yes. But that was different, because subversion had advantages for all user groups, not just a happy few that heavily use forward branches.

Also CVS development actively point to SVN as a successor, it was leaving a phased out (EOL) product. This is not the case with svn->GIT, and the tools are in many case a regression with git.

I wasn't there back then, so I can't say much about it. Speculative, maybe some people would have liked to stay on cvs, at least for some more time. They might not have spoken up, so how knows. But well speculation to be ignored....

Not sure about regressions, as I do not know what you refer too. Unless of course the need to adapt all scripts...

Quote

Quote
So did the change for fpc turn out to be good? Or would it be better, if fpc was still using cvs?

Definitely positive, but it still cost the project over an year, and while a bigger change in feature set, the amount of features used from the VCS was actually less. Little scripting and fixed procedures (e.g. for release and fixes branch maintenance).

So I expect it to be worse in this case, with smaller gains (which are also very unevenly distributed, and for some it might remain a netto minus).
Well, that I can not take away. It will take it's time. And for some of us, be a bigger learning curve....

And I can not predict, who benefits, and who not  -> If I could do that, I would have a booming business. :)

I can say, that I already benefited. I have for the past year (or year and a half) run my own 2-way mirror for Lazarus, and used it as if it was git. It has helped me a lot, and certainly saved me some time on my fpdebug work. (I.e. I used local commits as savepoints, and when I needed to recover from a mistake changing lots of sources, that saved me time / also it allowed me to compare before/after easily)
But I must also admit, that I spent a good 2 month of hard learning and falling (though I did not have anyone to ask, I had to google everything => so I hope that given that here we can support each other, learning time will be less)

Though, yeah, that does not help you.

Quote
Neither of those cases are now the case. The windows tooling is a step back, the commands are more complicated, the preparation less. And most importantly, SVN was not EOL.
I am trying to help with getting from svn to git. But yes, time is quite short. And I have other stuff too.

EOL has worked well for me, using git.

On Windows:
Code: Bash  [Select][+][-]
  1. git config core.autocrlf true

And there is .gitattributes, where you can make settings per file extension, folder, some kind of pattern, or individual file.
But I would need to lookup the details.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on June 24, 2021, 05:30:29 pm
more “modern” tools like Git
This is not helpful.

There were many reasons (I wont go into details) to switch (and there would have been some reasons not to, but well it went as it went).
This is certainly not a hype driven move.

However, if people like it, because it goes along with their favourite hype, they are welcome.


Quote
It’s just that some people like to force their world view, their ideology on others.
It's a lot more complicated than this. But this is not for here to be discussed.
And that is not that I want to suppress the discussion. Heck, personally I'd open a special sub-board for this... But we have a topic, and it is Pascal. (or in this thread svn/git)

So please note, that I will moderate the discussion, simply so we can focus on the actual topic.

PM, me if any of this needs clarification. Thanks.

As for the branch naming, I suggest the individual teams can discuss that on their private mail lists.
Title: Re: FPC & Lazarus moving to gitlab
Post by: marcov on June 24, 2021, 05:47:30 pm
but also to reduce commands to single lines again, since git seems to often need multiple commands with options for simple operations.
You can configure git aliases.
Code: Bash  [Select][+][-]
  1. git config alias.slog 'log --oneline'
  2. git slog -n 3

I know, but that is a dirty hack/workaround at best, doesn't fix the problem (requires configuration, complex in tutorials etc).

Quote
Quote
Yes. But that was different, because subversion had advantages for all user groups, not just a happy few that heavily use forward branches.

Also CVS development actively point to SVN as a successor, it was leaving a phased out (EOL) product. This is not the case with svn->GIT, and the tools are in many case a regression with git.
I wasn't there back then, so I can't say much about it. Speculative, maybe some people would have liked to stay on cvs, at least for some more time. They might not have spoken up, so how knows. But well speculation to be ignored....

Yes, there were. Roughly the same group of more occasional users.  One could argue then it was the EOL status of CVS, now it is the infrastructure decision.

Quote
Not sure about regressions, as I do not know what you refer too. Unless of course the need to adapt all scripts...

Regression in user experience (tortoisegit compared to tortoisesvn), single vs multi command etc.

Quote
Well, that I can not take away. It will take it's time. And for some of us, be a bigger learning curve....

A gigantic one, since now work and other projects aren't changing with it. IOW it must all be done in FPC time.

Quote
I can say, that I already benefited. I have for the past year (or year and a half) run my own 2-way mirror for Lazarus, and used it as if it was git. It has helped me a lot, and certainly saved me some time on my fpdebug work. (I.e. I used local commits as savepoints, and when I needed to recover from a mistake changing lots of sources, that saved me time / also it allowed me to compare before/after easily)

That is roughly the same scenario as for the compiler devels, deep into one subject/module.
Contrary to that I work all over the libraries, improving compatibility, small bugs, and doing the release branch work. I also work on multiple machines, which make local branches less useful, unless you create a complete shadow VCS.

Quote
But I must also admit, that I spent a good 2 month of hard learning and falling (though I did not have anyone to ask, I had to google everything => so I hope that given that here we can support each other, learning time will be less)

Though, yeah, that does not help you.

I don't have the time, so I gave up my central tasks. Now I'm just going to be bit player and committer, and we'll see in one or two years if I want to bear responsibility for something central again.

Quote
EOL has worked well for me, using git.

EOL as in CVS being end-of-life.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on June 24, 2021, 06:07:15 pm
Contrary to that I work all over the libraries, improving compatibility, small bugs, and doing the release branch work. I also work on multiple machines, which make local branches less useful, unless you create a complete shadow VCS.
Well, I don't know your workflow.

But again (and maybe there could be something for you in it) for me git did help, with testing my work on my VirtualMachines, and on my Laptop.

On my main working machine, I have my normal git clone.

I have then a git-server running, making that directory available as a server.
All my VM, and my laptop have that server as remote. (they have both, my local, and the true online remote)

So now I can commit (without pushing) either to the local default branch, or to a feature branch. Then I can pull those changes. If I make change I can push them back => though that needs a (temporary) feature branch.

So I no longer need shared folders, to move files or patches around. Or have my workindir entirely on a shared folder...


Quote
EOL as in CVS being end-of-life.
Ups...
Title: Re: FPC & Lazarus moving to gitlab
Post by: MarkMLl on June 24, 2021, 06:10:54 pm
I really don't want to prolong the agony but...

I mean, “we”, collectively as the FPC project, are (or should have been) aware that some contributors will stop contributing just because of that, while at the same time we ensure the project’s longevity by finally switching to more “modern” tools like Git.

I think one should be careful how one phrases that sentiment, since many among "the great unwashed" unashamedly judge Pascal by the received wisdom that "it's not modern".

Quote
Case in point “master” supposedly implied owning slaves. It’s rather fascinating that 99% of the English speaking world, i.e. first and second language English speakers mostly living outside of the US, don’t even understand the fuss about “master” yet we still have to put up with that shit, that language imperialism.

I am proud to say that I come from a family of schoolmasters.

MarkMLl
Title: Re: FPC & Lazarus moving to gitlab
Post by: avra on June 24, 2021, 06:19:59 pm
< SARCASM MODE ON > :-[ :-[ :-[
If I remember well, many have complained that much more developers would contribute if GIT was the main version control system. Can't wait for that to happen...
< SARCASM MODE OFF > ;)  ;)  ;)

I wonder what was the motivation for choosing gitlab over github? Self hosting possibility? Non M$? Code open sourced and available? Free storage size? Other?

Some guide will be needed for smoother crossing from Mantis - explaining several best practice use cases with common category/label mapping. Otherwise everyone will have his own idea on how to achieve that...
Title: Re: FPC & Lazarus moving to gitlab
Post by: MarkMLl on June 24, 2021, 06:28:39 pm
I have then a git-server running, making that directory available as a server.
All my VM, and my laptop have that server as remote. (they have both, my local, and the true online remote)

Etc. Again, I for one have never denied that Git is at least adequate (most would say better than adequate) as a tool for collaborative development.

My indignation is focussed on being obliged to embrace it in order to continue using Trunk and sporadically raise issues, which I consider to be unreasonable.

MarkMLl
Title: Re: FPC & Lazarus moving to gitlab
Post by: marcov on June 24, 2021, 06:32:55 pm
Well, I don't know your workflow.

Well, neither do I atm ;), since fixes branch and releases were a big part of my workload. But I'm sure gitlab will all automate that.

What you describe I also figured out for myself (mirror on a rpi4 with SSD, with a VPN thrown in for remote access to the mirror server for weekends and from work), but with my remaining commits mostly not related much to each other, creating, learning and maintaining such a shadow infrastructure is simply not worth it. 
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on June 24, 2021, 06:43:46 pm
Some guide will be needed for smoother crossing from Mantis - explaining several best practice use cases with common category/label mapping. Otherwise everyone will have his own idea on how to achieve that...
We are in the process, of importing *all* existing issues. During that process each team will decide what labels to use.
Title: Re: FPC & Lazarus moving to gitlab
Post by: avra on June 25, 2021, 01:26:48 am
Some guide will be needed for smoother crossing from Mantis - explaining several best practice use cases with common category/label mapping. Otherwise everyone will have his own idea on how to achieve that...
We are in the process, of importing *all* existing issues. During that process each team will decide what labels to use.
Thanks for that info. I have 2 new packages to add to FPC (can and bithelpers). Would it be better to provide patches now to Mantis or wait a little and do it in GitLab issues? I do not want to wait too much and miss FPC 3.4.0...
Title: Re: FPC & Lazarus moving to gitlab
Post by: edwinyzh on June 25, 2021, 06:30:52 am
I wonder what was the motivation for choosing gitlab over github? Self hosting possibility? Non M$? Code open sourced and available? Free storage size? Other?

Some guide will be needed for smoother crossing from Mantis - explaining several best practice use cases with common category/label mapping. Otherwise everyone will have his own idea on how to achieve that...
I have the same curiosity. But using git is definitely a great improvement!
Title: Re: FPC & Lazarus moving to gitlab
Post by: PascalDragon on June 25, 2021, 09:04:18 am
Quote
But, since the "svn advantage is gone", that can be done in GIT too. I can display the filtered log. I can
see the list, and pick the middle.

All my scripts (mostly FPC TProcess based) and every other infrastructure is invalidated. Worse, git seems to want bash for nearly everything which makes scripting a pain on Windows, specially when multiple driveletters are involved.

I have not once used bash on Windows to work with Git. Keep in mind that the Windows devs themselves are using Git nowadays (and before WSL!), so they're interested in a transparent integration as well.

I'm reluctant to do this all again, in case the gitlab thing doesn't work out in a few years, and we go back to hosted.

We'd probably roll our own instance of GitLab then. But the local commands wouldn't differ if the remote site is GitLab, GitHub or some plain Git repository.

< SARCASM MODE ON > :-[ :-[ :-[
If I remember well, many have complained that much more developers would contribute if GIT was the main version control system. Can't wait for that to happen...
< SARCASM MODE OFF > ;)  ;)  ;)

Yeah... that's a point I don't really believe in either. After all even if users should decide to provide pull/merge requests instead of merely patches we would still make sure that they are up to our standards which includes coding style and which will then even include how nicely done their commit history is (which there isn't with patches), so that there are no commits like "forgot to commit file x" or "fixed compilation".

I wonder what was the motivation for choosing gitlab over github? Self hosting possibility? Non M$? Code open sourced and available? Free storage size? Other?

Both "self hosting possibility" and "code open sourced and available".
Title: Re: FPC & Lazarus moving to gitlab
Post by: MarkMLl on June 25, 2021, 09:55:05 am
We'd probably roll our own instance of GitLab then. But the local commands wouldn't differ if the remote site is GitLab, GitHub or some plain Git repository.

Which I suppose raises the interesting question of whether anybody has already implemented a shim which implements Subversion checkout behaviour on top of Git.

MarkMLl
Title: Re: FPC & Lazarus moving to gitlab
Post by: ccrause on June 25, 2021, 12:07:07 pm
We'd probably roll our own instance of GitLab then. But the local commands wouldn't differ if the remote site is GitLab, GitHub or some plain Git repository.

Which I suppose raises the interesting question of whether anybody has already implemented a shim which implements Subversion checkout behaviour on top of Git.

MarkMLl
The opposite (git support for svn (https://git-scm.com/docs/git-svn)) does exist, but isn't helpful in this context... I suspect the problem is mapping from more complex to less complex functionality.  There is a kind of cheat sheet in the wiki that maps some common SVN operations to git equivalents (https://wiki.lazarus.freepascal.org/FPC_git#Common_SVN_operations_in_Git).  Not the same as remapped commands, but a good cross-over guide.
Title: Re: FPC & Lazarus moving to gitlab
Post by: DonAlfredo on June 25, 2021, 12:11:54 pm
Fpcupdeluxe extracts the SVN repo number from GIT, if it is stored.
Code: https://github.com/LongDirtyAnimAlf/fpcupdeluxe/blob/master/gitclient.pas#L563
Title: Re: FPC & Lazarus moving to gitlab
Post by: PascalDragon on June 25, 2021, 01:06:48 pm
We'd probably roll our own instance of GitLab then. But the local commands wouldn't differ if the remote site is GitLab, GitHub or some plain Git repository.

Which I suppose raises the interesting question of whether anybody has already implemented a shim which implements Subversion checkout behaviour on top of Git.

GitHub provides the ability to check out with SVN. So once we have a GitHub mirror of the GitLab repository that can be used.

Fpcupdeluxe extracts the SVN repo number from GIT, if it is stored.
Code: https://github.com/LongDirtyAnimAlf/fpcupdeluxe/blob/master/gitclient.pas#L563

There won't be any SVN revisions with the translated repositories and even if there were there won't be any new ones after the date of the switch.
Title: Re: FPC & Lazarus moving to gitlab
Post by: MarkMLl on June 25, 2021, 01:20:37 pm
GitHub provides the ability to check out with SVN. So once we have a GitHub mirror of the GitLab repository that can be used.

I might like the sound of that. Will that mean that going via Github there will still be something that approximates the current sequential release numbers?

As I've said, my beef is /entirely/ with the necessity of trunk users being pushed onto a new toolset.

MarkMLl
Title: Re: FPC & Lazarus moving to gitlab
Post by: DonAlfredo on June 25, 2021, 01:33:21 pm
Quote
There won't be any SVN revisions with the translated repositories and even if there were there won't be any new ones after the date of the switch.
For sure. But it might perhaps be good to consider a GIT commit hook script to magically create a kind of revision number or sequence that is a bit more human readable than the normal GIT hashes. And can be used as a reference.
Title: Re: FPC & Lazarus moving to gitlab
Post by: avra on June 25, 2021, 01:58:21 pm
consider a GIT commit hook script to magically create a kind of revision number or sequence that is a bit more human readable than the normal GIT hashes. And can be used as a reference.
+1
Title: Re: FPC & Lazarus moving to gitlab
Post by: dbannon on June 25, 2021, 02:34:30 pm
https://stackoverflow.com/questions/28792784/why-does-git-use-a-cryptographic-hash-function

The hash is a hash of the data in the repo at that point in time.  Its done to be a) unique and b) to guarantee the data.  SVN's sequential number does a) but tells us nothing about the data. On the other hand, git, while good at b)  cannot quite guarantee a).    :D

Personally, I have never taken a hash of my version to compare to a remote version of any git repo but I suppose someone does !

You could, I guess, have a layer that maps a sequential 'r' number to its hash. But you would need to rewrite, perhaps, half the git commands to do so.

I am a big fan of git but agree the svn 'r' number is heaps easier to use. (In most cases, you only use the first 6 to 8 digits of the hash).

Davo

 
Title: Re: FPC & Lazarus moving to gitlab
Post by: MarkMLl on June 25, 2021, 04:10:13 pm
The hash is a hash of the data in the repo at that point in time.  Its done to be a) unique and b) to guarantee the data.  SVN's sequential number does a) but tells us nothing about the data. On the other hand, git, while good at b)  cannot quite guarantee a).    :D

You could almost say that there are three numbering requirements:

a) Uniqueness guarantee

b) Integrity guarantee

c) Sequential ordering guarantee

I don't know whether the identifier that is yielded as a result of (c) really has to be a pure number, as long as it can be parsed as being sequential: while I routinely incorporate Subversion revision numbers into build identifiers I'm used to having e.g. 12345M and would be surprised if anybody tried manipulating them under scripted or program control.

Having said which and bearing in mind a suggestion (admission?) that somebody made in a PM, one possibility would be if an Svn-revision-surrogate actually showed a Git hash plus the number of subsequent commits... it might not be completely robust under all possible scenarios but it would allow a subteam investigating e.g. a trunk regression to say "we're at #1a2b3c4d.56, but if we revert 10 commits to #1a2b3c4d.46 the problem doesn't occur".

In some ways that mirrors some of the comments I've made in the past relating to conferencing systems, where it's enormously easier to thread and xref messages if message identifiers are guaranteed consistent from the POV of all users: SMF (i.e. like this forum) complies with that, Usenet doesn't unless it can be guaranteed that all messages are stored on a single locally-accessible server. Also some of https://en.wikipedia.org/wiki/Tumbler_(Project_Xanadu) might be relevant.

MarkMLl
Title: Re: FPC & Lazarus moving to gitlab
Post by: DonAlfredo on June 25, 2021, 04:27:34 pm
Example of post-commit (far from perfect):
Code: Pascal  [Select][+][-]
  1. #!/bin/bash
  2.  
  3. REVFILE='fpcgitrevision.txt'
  4.  
  5. exec 1>&2
  6. branch=`git rev-parse --abbrev-ref HEAD`
  7. shorthash=`git log --pretty=format:'%h' -n 1`
  8. revcount=`git rev-list HEAD --count`
  9. latesttag=`git describe --tags --abbrev=0  --always`
  10.  
  11. VERSION="$branch-$latesttag-$revcount-hash[$shorthash]"
  12. echo $VERSION > $REVFILE
  13.  
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on June 25, 2021, 05:30:32 pm
There is a kind of cheat sheet in the wiki that maps some common SVN operations to git equivalents (https://wiki.lazarus.freepascal.org/FPC_git#Common_SVN_operations_in_Git).
Actually in the spirit of GIT (more than one way for anything that may be done...), there are two: https://wiki.lazarus.freepascal.org/SVN_to_GIT_Cheatsheet
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on June 25, 2021, 05:31:37 pm
But it might perhaps be good to consider a GIT commit hook script to magically create a kind of revision number or sequence that is a bit more human readable than the normal GIT hashes. And can be used as a reference.
"git describe"

But tags need to be created yet
Title: Re: FPC & Lazarus moving to gitlab
Post by: DonAlfredo on June 25, 2021, 05:55:13 pm
Quote
But tags need to be created yet
Yep. Looked into this, but its not trivial to be honest. However, tags seem to be the only way to get something that resembles sequential release info that is a bit human readable. I can fully imagine/understand that this will not be implemented, but its worth investigating.
Title: Re: FPC & Lazarus moving to gitlab
Post by: DonAlfredo on June 25, 2021, 06:09:29 pm
In addition to the above.
As the current GIT repo of FPC trunk has the SVN release tags available, fpcupdeluxe allows its users to input a SVN revision. This revision will be searched for in the GIT logs, and converted into a GIT hash that is used to checkout the desired revision/commit.
So, the extra info that is put into future GIT-tags must be suitable for checkouts of the desired revision/commit. If not, tags are of no use IMHO.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on June 25, 2021, 06:10:43 pm
Quote
But tags need to be created yet
Yep. Looked into this, but its not trivial to be honest. However, tags seem to be the only way to get something that resembles sequential release info that is a bit human readable. I can fully imagine/understand that this will not be implemented, but its worth investigating.
I already did tests, should be easy.

But final details are not ready yet.
Title: Re: FPC & Lazarus moving to gitlab
Post by: af0815 on June 25, 2021, 07:11:02 pm
Question for Benchmarkig:
How did other git using projects handle this ? Eg. Linux-Kernel. Create a mimik from a svn workflow will complicate things in git.

Is this possible to use with a history. Eg. https://githowto.com/aliases and https://githowto.com/getting_old_versions . This produce a list with date and a shorten hash. The shorten hash with date can IMHO used like a tag inside a branch. If you use a master/develop modell you have a stable history for master and develop branch.

Short:
git config --global alias.hist "log --pretty=format:'%h %ad | %s%d [%an]' --graph --date=short"
or
git config --global alias.hist "log --pretty=format:'%h %ad | %s%d [%an]' --date=short"

git hist

The git hist show you nearly all you want without hacks
my 2 cents
Title: Re: FPC & Lazarus moving to gitlab
Post by: Bi0T1N on June 26, 2021, 12:20:35 am
As somebody who is not a core developer and has no (immediate) need to use Git or a Git-oriented hosting service for anything other than getting a current (.zip) snapshot, continued involvement with Trunk is tangential to everything else that I'm trying to do. In particular, I am not prepared to get myself into a situation where some needed facility of Trunk suddenly turns out to be broken, where if I want it to be fixed I've got to triage the release history myself, and where I find I can't easily revert to an older version because the .lpi or .lfm format has changed and I need a working Trunk to massage it back to a compatible format.
I don't see your problem. Instead of numbers you've hashes now but all of what you mentioned also works with hashes ;)
However, it would be nice if the FPC devs would consider appending the (short) commit hash to the output of the compiler version like its done now: Free Pascal Compiler version 3.3.1-r49285 [2021/04/28] for x86_64.

Contrary to that I work all over the libraries, improving compatibility, small bugs, and doing the release branch work. I also work on multiple machines, which make local branches less useful, unless you create a complete shadow VCS.
For doing small changes (stuff that don't really need to be tested like typos or other trivial things) Gitlab has the big advantage of the Web IDE (actually you can even compile there if it's setup). It is also possible to browse the code from the web browser (desktop and mobile).
And for the multiple machines, why don't you fork the project and push your changes to your user repository? Then you have your 'local' remote branch that can easily be shared (you only need internet access on all machines).


Well, what I really wanted to add is:
Code: Text  [Select][+][-]
  1. compiler/aarch64/ncpucnv.pas svneol=native#text/plain <- plain
  2. compiler/aarch64/ncpucon.pas svneol=native#text/pascal <- pascal
  3. compiler/aarch64/ncpuflw.pas svneol=native#text/pascal
  4. ...
  5. compiler/aarch64/ncpuset.pas svneol=native#text/plain <- plain
  6. compiler/aarch64/ra64con.inc svneol=native#text/plain
  7. compiler/aarch64/ra64dwa.inc svneol=native#text/plain
Title: Re: FPC & Lazarus moving to gitlab
Post by: MarkMLl on June 26, 2021, 09:54:11 am
As somebody who is not a core developer and has no (immediate) need to use Git or a Git-oriented hosting service for anything other than getting a current (.zip) snapshot, continued involvement with Trunk is tangential to everything else that I'm trying to do. In particular, I am not prepared to get myself into a situation where some needed facility of Trunk suddenly turns out to be broken, where if I want it to be fixed I've got to triage the release history myself, and where I find I can't easily revert to an older version because the .lpi or .lfm format has changed and I need a working Trunk to massage it back to a compatible format.
I don't see your problem. Instead of numbers you've hashes now but all of what you mentioned also works with hashes ;)

Read the full thread before commenting. Sequential numbering doesn't work.

MarkMLl
Title: Re: FPC & Lazarus moving to gitlab
Post by: PascalDragon on June 26, 2021, 10:45:41 am
  • It might be better to assign the created issues to the appropriate milestone instead of adding labels like Version 3.2.0. I mean you've the milestones with the versions but don't assign any issues. The label is no longer required.

The versions are the milestones, we don't consider anything else.

  • Providing a editorconfig (https://editorconfig.org/) file would help to have less problems with indentation and so on of people contributing code

Different parts of the project use different coding styles (e.g. compiler vs. RTL vs. packages). And this will not change.

  • The .gitignore file could be shrinked by using wildcards as this would simplify much of it: e.g. with *.exe *.o *.ppu all (sub)directories are automatically included

We have excluded those files and directories by experience. It's not enough to exclude selected files, because what is placed in e.g. the units directory might differ: If I should test something that outputs an additional file during compilation I don't want the whole bunch of units directories to pop up in git status.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on June 26, 2021, 11:53:54 am
  • The .gitignore file could be shrinked by using wildcards as this would simplify much of it: e.g. with *.exe *.o *.ppu all (sub)directories are automatically included

We have excluded those files and directories by experience. It's not enough to exclude selected files, because what is placed in e.g. the units directory might differ: If I should test something that outputs an additional file during compilation I don't want the whole bunch of units directories to pop up in git status.

1) For those who are not aware. Private ignores, that do not need to be commited in .gitignore
./.git/info/exclude

2) I do not know what will go in there. But again there may be interesting patterns

On by private git, I do use

**/test/**/*.log

** in this dir, or any subdir

Therefore, in any test dir (any where deep inside the workdir)
Ignore any file (nested dir or not), that has a .log extension
Title: Re: FPC & Lazarus moving to gitlab
Post by: Bi0T1N on June 26, 2021, 01:43:25 pm
As somebody who is not a core developer and has no (immediate) need to use Git or a Git-oriented hosting service for anything other than getting a current (.zip) snapshot, continued involvement with Trunk is tangential to everything else that I'm trying to do. In particular, I am not prepared to get myself into a situation where some needed facility of Trunk suddenly turns out to be broken, where if I want it to be fixed I've got to triage the release history myself, and where I find I can't easily revert to an older version because the .lpi or .lfm format has changed and I need a working Trunk to massage it back to a compatible format.
I don't see your problem. Instead of numbers you've hashes now but all of what you mentioned also works with hashes ;)

Read the full thread before commenting. Sequential numbering doesn't work.
I did and I explained to you that it does but just in the 'git' way where numbers == hashes. You can't expect to get the same output with a complete different principle. Don't try to see everything through a 'svn' view.

  • It might be better to assign the created issues to the appropriate milestone instead of adding labels like Version 3.2.0. I mean you've the milestones with the versions but don't assign any issues. The label is no longer required.

The versions are the milestones, we don't consider anything else.
Not sure if you understood what I meant. If you go to the Milestones (https://gitlab.com/freepascal.org/fpc/testconversion2/-/milestones), you'll notice that no issues are assigned to them. Instead every issue has a label that indicates the version it belongs to. That's not really following the general 'Gitlab principle'.

  • Providing a editorconfig (https://editorconfig.org/) file would help to have less problems with indentation and so on of people contributing code

Different parts of the project use different coding styles (e.g. compiler vs. RTL vs. packages). And this will not change.
Didn't said that but some units e.g. might use tab size of 2 and others using 4. This could be individually configured with the EditorConfig.

  • The .gitignore file could be shrinked by using wildcards as this would simplify much of it: e.g. with *.exe *.o *.ppu all (sub)directories are automatically included

We have excluded those files and directories by experience. It's not enough to exclude selected files, because what is placed in e.g. the units directory might differ: If I should test something that outputs an additional file during compilation I don't want the whole bunch of units directories to pop up in git status.
If you scroll through the .gitignore file you should notice that it always list the file extensions .exe, .o and .ppu which is kinda nonsense as you could use wildcards which will block them as well. If you want to commit and track single files of these types, just add this explicitly to the gitignore.
Title: Re: FPC & Lazarus moving to gitlab
Post by: marcov on June 26, 2021, 02:02:49 pm
( I pile it on a bit to make the point that not everybody sees this as an improvement)

For doing small changes (stuff that don't really need to be tested like typos or other trivial things) Gitlab has the big advantage of the Web IDE (actually you can even compile there if it's setup). It is also possible to browse the code from the web browser (desktop and mobile).

FPC has had a compile test before commiting (now pushing I assume) for a long time. But it would be useful for things like docs,  readmes etc.

But it is not a solution to anything development related. I can understand this if you are collaborating on a 50 lines script with a friend next door, but I do hope most projects have rules about making sure that main is always compiling. CI/Jenkins should only be to catch the leftover mistakes, not laziness.

(that said, in a work environment, CI solutions to check compiling work much easier, but that is because the guy breaking it is usually easily contacted, in the same room/building or at least at the keyboard for the rest of the working day, it is different with 24x7 projects done in spare time)

Quote
And for the multiple machines, why don't you fork the project and push your changes to your user repository? Then you have your 'local' remote branch that can easily be shared (you only need internet access on all machines).

So that I have push everything twice? So when I complain that  add+push all the time is more complicated than "commit" you suggest a solution with add+push+push ? Sigh.

You can have own repos on a rpi with branching etc with VPN, but new software is usually about improving workflow and reducing configuration and maintenance. Not doubling the whole infrastructure at the first workaround

Not about creating and maintaining a complete shadow server infrastructure, and even then clicking through 20 freaking questions while installing tortoisegit+git separately (tortoisesvn unified and only 3 questions) and even then having to do additional "git config"s.

Stone age junk software.

Quote
  • It might be better to assign the created issues to the appropriate milestone instead of adding labels like Version 3.2.0. I mean you've the milestones with the versions but don't assign any issues. The label is no longer required.

It would be nice. If milestones were actually clear up front. But in practice what goes into a fixes branch is often only finally decided in the weeks before branching. This because many people work on features in their own pace, which makes it hard to plan due to both technical and real life setbacks. 

Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on June 26, 2021, 02:36:38 pm
Lets just establish a bit common ground here.

To use git, one has to break with old habits. And "old habits die hard".

The value of doing that, depends on what you get back.
I am an experienced git user (and I feel very confident to put it that way).
Yet, I have always and totally independent of this current discussion, stood the point: There are use cases where svn is the better choice for a person, or even a project.
And yes, I am taking into account all the fancy stuff that git offers. In some situation that does not add value.

"The binford 6100 is not needed to hang a picture"

Of course all the above, is not really helpful here.
The projects move. That's a done fact.
But just because people have to live with that, does not mean we need to convince them they are wrong. We may try to explore if they have sufficient information for their judgment, and offer it. But once we done that, we should have the capacity to accept an individuals judgment about their own situation in this.


Also, I have put a lot of time, into the specifics of moving to git for Lazarus. And there are many details that need very close attention being paid to them.
It's not a "one weekend to move to git" conversion. It's a lot more under the hood.

I wont go into all the details. But it's a lot. And it does affect everyone in the team.

Maybe in a year, we will know if we did a good job. And hopefully we will have done.
Title: Re: FPC & Lazarus moving to gitlab
Post by: af0815 on June 26, 2021, 03:44:22 pm
Lets just establish a bit common ground here.

To use git, one has to break with old habits. And "old habits die hard".
+1
Title: Re: FPC & Lazarus moving to gitlab
Post by: MarkMLl on June 26, 2021, 04:34:00 pm
Lets just establish a bit common ground here.

To use git, one has to break with old habits. And "old habits die hard".
+1

Absolutely. But breaking habits can take resources, and on occasion one has to make decisions based on cost vs benefit.

MarkMLl
Title: Re: FPC & Lazarus moving to gitlab
Post by: PascalDragon on June 27, 2021, 12:57:03 pm
  • It might be better to assign the created issues to the appropriate milestone instead of adding labels like Version 3.2.0. I mean you've the milestones with the versions but don't assign any issues. The label is no longer required.

The versions are the milestones, we don't consider anything else.
Not sure if you understood what I meant. If you go to the Milestones (https://gitlab.com/freepascal.org/fpc/testconversion2/-/milestones), you'll notice that no issues are assigned to them. Instead every issue has a label that indicates the version it belongs to. That's not really following the general 'Gitlab principle'.

And who said that we care about the "Gitlab principle"? We are doing this as we see fit and not how some website thinks is the way it should work. If we'll never use these milestones then we'll simply never use them.

  • Providing a editorconfig (https://editorconfig.org/) file would help to have less problems with indentation and so on of people contributing code

Different parts of the project use different coding styles (e.g. compiler vs. RTL vs. packages). And this will not change.
Didn't said that but some units e.g. might use tab size of 2 and others using 4. This could be individually configured with the EditorConfig.

If you consider it so important then provide patches for it.

  • The .gitignore file could be shrinked by using wildcards as this would simplify much of it: e.g. with *.exe *.o *.ppu all (sub)directories are automatically included

We have excluded those files and directories by experience. It's not enough to exclude selected files, because what is placed in e.g. the units directory might differ: If I should test something that outputs an additional file during compilation I don't want the whole bunch of units directories to pop up in git status.
If you scroll through the .gitignore file you should notice that it always list the file extensions .exe, .o and .ppu which is kinda nonsense as you could use wildcards which will block them as well. If you want to commit and track single files of these types, just add this explicitly to the gitignore.

It turned out (after we had a similar discussion on the core mailing list) that I looked at the .gitignore file in the SVN repository and not the one contained in the conversions as I consider the former the master. And there it's a much more compact file. I agree that what was generated by the conversion (by using the svn:ignore properties) is bad and in my opinion we should stick with the hand craft one.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on June 27, 2021, 04:47:13 pm
Does this mean that it will no longer be possible to refer to a particular stage of trunk development as "revision such-and-such", and to step forwards and backwards from that point using a contiguous sequence of revision numbers?
Mainly, yes.

The exact details on the below are still being evaluated. Outcome not yet known....

Ok, we now have an example of what it will likely be for the Lazarus repro (might still get some amendments, but should be close).
https://gitlab.com/freepascal.org/lazarus-ide/lazarus_test_conversion_2

Code: Bash  [Select][+][-]
  1. git describe
will return output like this
Code: Text  [Select][+][-]
  1. trunk-2_3-59-gd9603ca06a
  2. lazarus_2_0_8-144-g532537c6c4

This are 3 fields
* The name of a label
* The amount of commits from that label (everything git log would see for that range)
* The short hash-id (for access to the exact commit)

You may notice that the example in the fixes branch is based on 2.0.8, even though I can tell you the commit for this is past 2.0.12.
The reason is that 2.0.12 "branched" from the fixes branch. Therefore the 2.0.12 label was not found.
But the general idea, of having some sort of "sortable" reference is still valid.

However those numbers are not 100% unique.
When branches divert (e.g. pulled in merge requests / or new branches without tag at their base), then both lines of the branch use the same numbers.
That is usually only a small amount of commits. So you still can tell a rough relation between 2 describes.
After the merge, when the branch continues as one, each commit has only ONE number, (even if the 2 branch line differed in commit count)
Title: Re: FPC & Lazarus moving to gitlab
Post by: MarkMLl on June 27, 2021, 09:43:37 pm
Code: Bash  [Select][+][-]
  1. git describe
will return output like this
Code: Text  [Select][+][-]
  1. trunk-2_3-59-gd9603ca06a
  2. lazarus_2_0_8-144-g532537c6c4

This are 3 fields
* The name of a label
* The amount of commits from that label (everything git log would see for that range)
* The short hash-id (for access to the exact commit)

You may notice that the example in the fixes branch is based on 2.0.8, even though I can tell you the commit for this is past 2.0.12.
The reason is that 2.0.12 "branched" from the fixes branch. Therefore the 2.0.12 label was not found.
But the general idea, of having some sort of "sortable" reference is still valid.

However those numbers are not 100% unique.

Definitely getting there though, congratulations.

Do you envisage some way of doing a binary chop based on that reference?

MarkMLl
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on June 27, 2021, 10:55:17 pm
Do you envisage some way of doing a binary chop based on that reference?

Bisect? No.

First of all this is for estimating where are reporter is at. (at least the tag + number)
Someone tells me his IDE about says svn rev 34009 => I ask where he was the last 10 years.
Same for "trunk-0_9_30-200-HASH"


Assume this branch, with a merge
Code: Text  [Select][+][-]
  1.     / - A - B - C \
  2. tag                   - E - F
  3.     \ - D -      /
  4.  
A and D both are "tag + 1"
E  is at "tag + 5" (all 4 commits are seen before it)

And there wont be a tag on every little merge branch. (Even though merges will be for features, pull requests ... otherwise we hopefully keep it linear)

However similar issues in svn. You look at r300. You do not know if r301 is in a different branch.


Code: Pascal  [Select][+][-]
  1. lazarus_2_0_8-144-g532537c6c4

Git has no function that takes the  "lazarus_2_0_8-144" part and computes a  commit (it can't / svn can, but you may still end in a diff branch). You can compute it for git, by getting the list of all commits between the start and end point, and then get the element from the list, but that is not what you want?

To see the commit, you need the hash part, that is why it is included.

If you have the HASH, you can go n parents towards the root.
Code: Text  [Select][+][-]
  1.   g532537c6c4~1
  2.   g532537c6c4~2
  3.   g532537c6c4~3
But you cannot walk children towards the current most recent commit.


Code: Text  [Select][+][-]
  1. git rev-list   g532537c6c4..HEAD
gives you a list of all commits from (exclusive)   g532537c6c4 to (inclusive) head. (current)

Code: Text  [Select][+][-]
  1. git rev-list  --bisect  g532537c6c4..HEAD
(not tested) gets you the one in the middle

Code: Text  [Select][+][-]
  1. git log --oneline g532537c6c4..HEAD
gets you the log to look on



before I go into more.
Describe more closely how you bisect in svn.

Because either you use all commits (and hope no numbers in that range are in a branch) => and then you do (r1 + r2) div 2 => but then you can do git bisect. (or rev-list --bisect)

Or you do something else (pre-select commits), but the svn rev numbers are just from a list with gaps, no longer contineous, and all math is gone.
If there are gaps in your svn list, how to you get the commit + 60 ? 

Yes, the svn numbers are easier to copy. You  look at them, and you can memorize them just until you typed them.
The git hashes => copy and paste.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Okoba on June 29, 2021, 11:51:14 pm
Thank you team, especially Martin  ;D
Title: Re: FPC & Lazarus moving to gitlab
Post by: Okoba on July 01, 2021, 12:06:19 pm
@Martin_fr Suggestion: Removing "-IDE" from Lazarus group name. It eases daily work like copying and pasting a link, project name or folder of the project and generally it seems redundant. I know the website is called Lazarus-IDE, but it seems fine to have only Lazarus as the name of the group and project, especially as you've created Lazarus in the FPC group, so no misunderstanding seems likely.
Title: Re: FPC & Lazarus moving to gitlab
Post by: valdir.marcos on July 01, 2021, 04:35:44 pm
Does this mean that it will no longer be possible to refer to a particular stage of trunk development as "revision such-and-such", and to step forwards and backwards from that point using a contiguous sequence of revision numbers?

MarkMLl
We could use Firebird's build number schema via CI/CD [Continuous Integration and Continuous Delivery]:
https://github.com/FirebirdSQL/firebird/branches
https://github.com/FirebirdSQL/firebird/commits/master
increment build number [via CI/CD]:
https://github.com/FirebirdSQL/firebird/commit/6bf7834d30a28d0624bc12c0a28692c7724f99ca
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on July 01, 2021, 09:32:54 pm
@Martin_fr Suggestion: Removing "-IDE" from Lazarus group name.
I've changed the URL.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Okoba on July 01, 2021, 09:46:16 pm
What URL?
I meant Gitlab project name for Lazarus.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on July 01, 2021, 10:30:34 pm
What URL?
I meant Gitlab project name for Lazarus.
The group name itself can afaik not be changed (other than starting from scratch / no thanks). But that is a display name. So it should not be a problem. (if you copy and paste it as "lazarus" people will still know)

The URL is now
https://gitlab.com/groups/freepascal.org/lazarus
And the test repro is at
https://gitlab.com/freepascal.org/lazarus/lazarus_test_conversion_2

both without the "-ide".




EDIT: it can be changed. Just not in the advanced settings.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Okoba on July 01, 2021, 10:55:10 pm
It was not updated the previous check but now it is Lazarus.
Thank you for the update!
Project names will be FPC and Lazarus like the group names?
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on July 01, 2021, 11:03:47 pm
It was not updated the previous check but now it is Lazarus.
Thank you for the update!
Project names will be FPC and Lazarus like the group names?

Lazarus has a 2 repositories. It's likely the main one will end up as just Lazarus. (the other is for binaries, gdb,qt... / there is a test of it too).

I don't know how fpc is going to handle their build repro and all...
Title: Re: FPC & Lazarus moving to gitlab
Post by: Okoba on July 01, 2021, 11:05:57 pm
Fantastic. And again thank you for the great work.
I will be happy if I can help in any way but you have my best wishes anytime.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Zaher on July 08, 2021, 11:59:22 am
It is a bad news for who live underground world, like me.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on July 08, 2021, 01:06:18 pm
It is a bad news for who live underground world, like me.
Meaning?
Title: Re: FPC & Lazarus moving to gitlab
Post by: Zaher on July 08, 2021, 01:21:15 pm
ah, gitlab blocking some countries, Syria one of ot because it is based on google services.

https://twitter.com/compumaster/status/1029164513856110593

https://about.gitlab.com/blog/2018/07/19/gcp-move-update/?utm_medium=social&utm_source=twitter
Title: Re: FPC & Lazarus moving to gitlab
Post by: PascalDragon on July 08, 2021, 01:25:35 pm
ah, gitlab blocking some countries, Syria one of ot because it is based on google services.

https://twitter.com/compumaster/status/1029164513856110593

https://about.gitlab.com/blog/2018/07/19/gcp-move-update/?utm_medium=social&utm_source=twitter

And that's why I am in favour of self-hosting.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Fred vS on July 08, 2021, 01:31:52 pm
ah, gitlab blocking some countries, Syria one of ot because it is based on google services.

https://twitter.com/compumaster/status/1029164513856110593

https://about.gitlab.com/blog/2018/07/19/gcp-move-update/?utm_medium=social&utm_source=twitter

IMHO, (maybe I am wrong) the plan of fpc+lazarus is to use the source of gitlab site but keep the data on their own server, not the server of Gitlab.inc.
Title: Re: FPC & Lazarus moving to gitlab
Post by: marcov on July 08, 2021, 01:34:02 pm
Yes and no, originally the plan was a own hosted gitlab instance, now the plan is move to the gitlab cloud.

Title: Re: FPC & Lazarus moving to gitlab
Post by: Fred vS on July 08, 2021, 01:37:55 pm
Yes and no, originally the plan was a own hosted gitlab instance, now the plan is move to the gitlab cloud.

Ha, ok. (but it would be good to keep the idea to host git data on your own server when all is working ok with gitlab cloud).
Title: Re: FPC & Lazarus moving to gitlab
Post by: MarkMLl on July 08, 2021, 01:43:07 pm
And that's why I am in favour of self-hosting.

I echo that sentiment. But jurisdiction is a problem which is not going away, and I suspect will only get more complex as collaborative projects get increasing attention.

The sort of thing that worries me is that if a developer in the UK contributed to a project on a server which he knew was mirrored in a jurisdiction which had no scruples serving content to (insert blacklisted country here), the security services would eventually come knocking on his door.

I'm not aware of that having happened yet. I am aware of a claim that somebody was treated very harshly after sending money to the family of his web developer who was killed by USA-allied fire in the Balkans.

MarkMLl
Title: Re: FPC & Lazarus moving to gitlab
Post by: 440bx on July 08, 2021, 01:47:17 pm
I am aware of a claim that somebody was treated very harshly after sending money to the family of his web developer who was killed by USA-allied fire in the Balkans.
Politics... the bug that cannot be debugged.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on July 08, 2021, 02:06:43 pm
The data will be kept on gitlab.

But mirrors can be added. (IIRC gitlab can even push updates to mirrors).

So a github (read only) mirror is likely.
Other mirrors (including self hosted) can be done, but I am not sure what the immediate plans are.

------------------
Pull requests (git / not github) can be made for any public git server.
Even if you host your own (or use some other git hoster).

A pull request (git) is simply an email (or other form of communication) that says please pull branch FOO from my git @ http://server/repro

And the person who receives the request then does
Code: [Select]
git pull http://server/repro FOO
Ideally that repro is a clone. Otherwise the pull retrieves the data, but the data is unrelated (i.e. no common ancestor)
git pull git@github.com:your-repo-ssh.git remote-branch-name

Title: Re: FPC & Lazarus moving to gitlab
Post by: Zaher on July 08, 2021, 02:12:06 pm
for me as developer I can access gitlab using vpn (until now),
the most important is download binaries, if you can put mirror on github, or any site that not censoring it, it is good idea, it is for normal developers in sanction countries.
Title: Re: FPC & Lazarus moving to gitlab
Post by: PascalDragon on July 09, 2021, 07:22:42 am
for me as developer I can access gitlab using vpn (until now),
the most important is download binaries, if you can put mirror on github, or any site that not censoring it, it is good idea, it is for normal developers in sanction countries.

Binaries will still be hosted by us directly or on SourceForge. We currently have no plans to change that.
Title: Re: FPC & Lazarus moving to gitlab
Post by: JulianFPC on July 17, 2021, 11:24:14 pm
There seems to be a problem with the IDs in recent bugtracker conversion. I only see @1234567 for the users who provided their ID in number form (including me). Will this be fixed? The other users who chose their user name seem to be working.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on July 17, 2021, 11:53:59 pm
Do you have a link to an example please?

--EDIT:
found, forwarded to relevant person
Title: Re: FPC & Lazarus moving to gitlab
Post by: dbannon on July 20, 2021, 05:44:50 am
The date for the final conversion has been established as the weekend of 17/18 july.

Not in anyway pushing, just curious. Is the any progress ?

Davo
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on July 20, 2021, 09:39:56 am
We moved it one week.
Title: Re: FPC & Lazarus moving to gitlab
Post by: FPK on July 24, 2021, 02:44:11 pm
And that's why I am in favour of self-hosting.

I echo that sentiment.

It is simply a matter of time: most of us prefer developing instead of doing system administration. Most system administration of FPC is done by the main code contributors, so this narrows it down that we go for external hosting now. Not to mention the fact that running a self-hosted server securely becomes more and more difficult during the last years not only technically but also legally, e.g. considering the GDPR.
Title: Re: FPC & Lazarus moving to gitlab
Post by: MarkMLl on July 24, 2021, 03:12:38 pm
Noting that Mantis has gone, but I'm disappointed that it wasn't at least temporarily replaced by a redirect or at least a sensible message.

MarkMLl
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on July 24, 2021, 04:31:55 pm
The plan is to install a redirect. (for each individual issue)

But that can only happen, once the target exists.
And may need some time for testing, preparing the data, etc.

For now, translation to gitlab is in progress (and will take some time)
Title: Re: FPC & Lazarus moving to gitlab
Post by: lucamar on July 28, 2021, 01:12:58 pm
I asked the same about Lazarus but since they are different projects, can anyone tell us what will happen with the repo at:
https://svn.freepascal.org/svn/fpc/
???
Title: Re: FPC & Lazarus moving to gitlab
Post by: marcov on July 28, 2021, 01:18:18 pm
I asked the same about Lazarus but since they are different projects, can anyone tell us what will happen with the repo at:
https://svn.freepascal.org/svn/fpc/
???

It will be combined with the pre 1998 CVS (which was not online) and also move to gitlab. With the whole merging and stuff there were some problems, so this is the repo that moves the slowest.
Title: Re: FPC & Lazarus moving to gitlab
Post by: prof7bit on July 30, 2021, 10:28:07 am
What will happen to old bugs in Mantis?

The forums are full of links like "yeah, this is a known bug, see here: <link to mantis>"

and clicking it now sends me to the gitlab login page. Will these bugs be archived somewhere to be read only but still accessible in some way?

--

I very much appreciate the move to git and actually have waited for this day for a long time already, but the disappearance of all old bugs is causing me some confusion.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on July 30, 2021, 10:38:28 am
Mantis will go away.

The URLs will keep working  as forwarder to gitlab.
Actually the forwarders are in place, but gitlab is not yet ready.

Title: Re: FPC & Lazarus moving to gitlab
Post by: prof7bit on July 30, 2021, 01:09:34 pm
Was it really necessary to take down the svn server before everything is in place on gitlab?

I have been using fpcupdeluxe for the last year or two to have easy access to various branches and updates and today I noticed it stopped working completely because svn is gone and on gitlab the project is still rapidly switching temporary names like "testconversion2" or "testconversion" which I assume will later be changed to something more permanent like "fpc", currently even the latest pre-release of fpcupdeluxe can't keep up with the changing repository urls.

Also when browsing through gitlab I cannot seem to find the Lazarus repository at all, there is only "Binaries", "CCR" and "Website"




Title: Re: FPC & Lazarus moving to gitlab
Post by: DonAlfredo on July 30, 2021, 01:22:54 pm
This is unfortunate. But please give the dev some time to sort things out.
As things are rapidly changing right now and a lot of work is done, I would suggest not to touch your current fpcupdeluxe install.
As up-maintainer, I am trying to follow the changes as good as possible. A new release will be available as soon as gitlab is up and running !
Title: Re: FPC & Lazarus moving to gitlab
Post by: prof7bit on July 30, 2021, 01:25:29 pm
I would suggest not to touch your current fpcupdeluxe install.

Fortunately I have btrfs and snapshots, otherwise I would not have a working Lazarus installation anymore now :-/.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on July 30, 2021, 01:36:59 pm
Was it really necessary to take down the svn server before everything is in place on gitlab?

It wasn't planned...

We had aimed to move on the 14th (start + 2 or 3 days for the move)

We've tested the move over and over in the past 2 month.
But despite this, some last minute issues came in. (not all technical, but some technical).

So we where forced to move it back by one week. (knowing that the server would go offline at the end of the month).
And then it took more than the 3 days planed...

Of course, even that would have left "only" 2 weeks for everyone to switch from svn to git.

Btw, Lazarus repro is avail on the Github mirror, including githubs "git to svn" svn access. (which you may be able to use in FpcUpDeluxe for the time being)



That said, retrospectively a lot could have done better. Better communicated too.
But, this isn't our normal line of work. So we didn't foresee every detail.
If we were starting the move now, with the knowledge we gained => that would be a diff story.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on July 30, 2021, 01:40:09 pm
I would suggest not to touch your current fpcupdeluxe install.

Fortunately I have btrfs and snapshots, otherwise I would not have a working Lazarus installation anymore now :-/.

Someone needs to check if FpcUpDeluxe works with SVN access to:
    https://github.com/fpc/Lazarus.git

Yes it says .git. But github supports svn on that url.

Mind, keep to tags => revision numbers will be diff from the existing, as this is translated back from the git repro.
(On the other hand, it even gets new commits)
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on July 30, 2021, 01:49:46 pm
Ok, Lazarus is available now at:

https://gitlab.com/freepascal.org/lazarus/lazarus.git
Title: Re: FPC & Lazarus moving to gitlab
Post by: Seenkao on July 30, 2021, 02:09:50 pm
День добрый!
Будет неплохо узнать причины переезда с SVN на GitLab. Хотя бы ради простого интереса и информации для пользователей.

Google translate:
Good afternoon!
It would be nice to know the reasons for moving from SVN to GitLab. At least for the sake of simple interest and information for users.
Title: Re: FPC & Lazarus moving to gitlab
Post by: DonAlfredo on July 30, 2021, 02:10:42 pm
Added a pre-release of fpcupdeluxe that will work with and without gitlab.
Please use with care.
https://github.com/LongDirtyAnimAlf/fpcupdeluxe/releases/tag/2.0.0a
Title: Re: FPC & Lazarus moving to gitlab
Post by: PascalDragon on July 30, 2021, 02:27:09 pm
It would be nice to know the reasons for moving from SVN to GitLab. At least for the sake of simple interest and information for users.

The switch to Git was intended for quite some time already. The switch to a hosting service such as GitLab was due to not wanting to maintain a local solution (though with GitLab we can in principle switch back to a local one if necessary). Due to a change in our server infrastructure it was decided to do this step now.
Title: Re: FPC & Lazarus moving to gitlab
Post by: stocki on August 01, 2021, 07:28:16 am
These changes annoy me. I've been doing this for the past 30 years:

svn co https://svn.freepascal.org/svn/fpc/trunk fpc
svn co https://svn.freepascal.org/svn/lazarus/trunk lazarus

What do I have to do with git?
Title: Re: FPC & Lazarus moving to gitlab
Post by: lucamar on August 01, 2021, 07:51:34 am
I've very limited knowledge of Git, so if this wrong, please somebody correct me but AFAICT, you need to do for the first time:
Code: [Select]
git clone https://gitlab.com/freepascal.org/lazarus/lazarus.git mylazarusdirand from then on,
Code: [Select]
git pullfrom "mylazarusdir"

Idem with FPC, once the final repo is set-up.

As Martin said in the opening post, some info pages have been set-up in the wiki to ease the transition:
All necessary information to connect to gitlab will be collected in the FPC &
Lazarus Wiki. Several pages have already been set up:

https://wiki.freepascal.org/FPC_git
https://wiki.freepascal.org/FPC_git_concepts
https://wiki.freepascal.org/SVN_to_GIT_Cheatsheet

These pages will be updated with the correct URLS when the final conversion happens. The FPC & Lazarus websites will also be adapted with new instructions.

Note, also, that you can still use SVN by changing the URL to the clone set-up in GitHub. I don't know off the top of my head which it's but it's cited somewhere in this thread.
Title: Re: FPC & Lazarus moving to gitlab
Post by: stocki on August 01, 2021, 09:03:51 am
I don't understand why the old SVN stopped working abruptly. I don't understand why partial git solutions are always presented. Very unprofessional.
Title: Re: FPC & Lazarus moving to gitlab
Post by: jshand2010 on August 01, 2021, 09:11:33 am
hi there.  i've also noticed there are teething problems with the snv to git convertion.  i have found that lazarus gitlab is setup without problems, but when i do a search for FPC, it doesn't exist...is there a reason for that
Title: Re: FPC & Lazarus moving to gitlab
Post by: Bart on August 01, 2021, 09:13:36 am
Very unprofessional.

We are not a professional organisation.
All of us are volunteers.
There are good reasons to move to a hosted solution (GitLab in this case).
Migrating everything, including the bugtracker to GitLab is a major operation and it takes time.
Martin has put many, many hours into it.

Yes, thing go wrong, so we try and fix them.
Yes, the transition takes time.
Yes, subversion repository needs to be read-only until git repo works correctly
Yes, that's annoying

But, complaining about that won't help at all.
The only thing you can achieve by this that volunteers like Martin feel it makes no sense putting their energy into this project anymore.

If you want a professional solution, then pay for one: buy Delphi and see where that takes you.

And, please do not compare the Lazarus project to e.g. the gcc community.
We have a small core team of developer, not hundreds like them.

Bart
Title: Re: FPC & Lazarus moving to gitlab
Post by: Bart on August 01, 2021, 09:14:52 am
hi there.  i've also noticed there are teething problems with the snv to git convertion.  i have found that lazarus gitlab is setup without problems, but when i do a search for FPC, it doesn't exist...is there a reason for that

Yes, there is (there were issues, but I forgot what exactly).
It'll come.

Bart
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on August 01, 2021, 10:34:28 am
hi there.  i've also noticed there are teething problems with the snv to git convertion.  i have found that lazarus gitlab is setup without problems, but when i do a search for FPC, it doesn't exist...is there a reason for that
As I said before...

The life time of the server came to an end. Since the underlying server was in bad need for maintenance, the contract for it had been cancelled. That was quite a while back. (From what I understand, I do/did not hold the contract).

The plan was to be done in time (on the 14th/15th).
But as things go, that did not work out.

We had to move the start of transferring to the weekend after that.

And then conversion took unplanned longer.

As you saw on the lazarus git, it was published and taken down again.
I had tested the process several times.
But only after we went live, I found that some older commit, well hidden had gone wrong. So I needed to start the Lazarus git over. It took Michael and me 3 days to get it back online.

The fpc repro has a lot more work to do (because they are recovering some of the data, they had lost when they moved from cvs to svn). (At least that is what I understand).

So unfortunately the old server went out of its live time, before we where ready.


And as explained by Bart: Neither of us is doing such type of tasks on a regular basis. All the planing, communication, testing .... All that was learning by doing.
So, yes, it was by no way perfect. And if we would have had all the experience (that we gained) when we started, then a lot would have been done differently.




Title: Re: FPC & Lazarus moving to gitlab
Post by: stocki on August 01, 2021, 11:17:11 am
Why do you have to switch from SVN to GIT now? SVN is highly professional and stable. Why these changes all the time? When a server is down, I buy a new one and copy everything. Finished.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on August 01, 2021, 11:21:18 am
Why do you have to switch from SVN to GIT now? SVN is highly professional and stable. Why these changes all the time? When a server is down, I buy a new one and copy everything. Finished.
Has been answered at least 3 times.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Bart on August 01, 2021, 11:53:10 am
Why do you have to switch from SVN to GIT now? SVN is highly professional and stable. Why these changes all the time? When a server is down, I buy a new one and copy everything. Finished.

This is the last time:

Because the small core team does not want to spent their energy on maintaining the server and fixing bugs in Mantis anymore.
Maintenance became a real PITA.
They want to spend their free time developing fpc/Lazarus.
Therefore a hosted solution was sought.
Most of the core developers prefer git over svn.
It is their prerogative to decide that.
(Personally I'm more comfortable with svn).

For non-developers who use trunk in stead of svn co/svn up, you now have git clone/git pull to obtain current sources.
Making diff's/patches is equally simple.

Most regular users however just use the binaries we publish, so for them it does not matter at all.

Again, and for the last time: feel free to abandon fpc/Lazarus and buy Delph or start using a free C-compiler if you're not willing to pay huge amounts of money for a compiler.
Otherwise, just go with the flow and see where it takes you.

We don't need this negative energy.

Bart
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on August 01, 2021, 12:14:02 pm
I've put together a wiki page on the subject. It's just from my memory. So probably missing various points (as the topic has been bespoken in the team for a long time)

https://wiki.lazarus.freepascal.org/FAQ_SVN_to_GIT
Title: Re: FPC & Lazarus moving to gitlab
Post by: dbannon on August 01, 2021, 04:16:01 pm
...
For non-developers who use trunk in stead of svn co/svn up, you now have git clone/git pull to obtain current sources.
Making diff's/patches is equally simple.
....
And if you don't want to use even that minimal git command, you can pull down a full zip or gz file in less time than the source forge ads ran for when grabbing a binary.

Davo
Title: Re: FPC & Lazarus moving to gitlab
Post by: Fred vS on August 01, 2021, 05:20:54 pm
...
For non-developers who use trunk in stead of svn co/svn up, you now have git clone/git pull to obtain current sources.
Making diff's/patches is equally simple.
....
And if you don't want to use even that minimal git command, you can pull down a full zip or gz file in less time than the source forge ads ran for when grabbing a binary.

Davo

I noted that strangely lot of people dont see that button in GitLab/GitHub site.
I have to explain with a screenshot where is that button ( even that in GitHub it is the only green button ).

Maybe a screenshot could be added in the wiki.

Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on August 01, 2021, 05:28:46 pm
Maybe a screenshot could be added in the wiki.
Depends on which wiki site.

There are still several pages to be updated from svn to git....

Also, pages should generally prioritize in the order:

gitlab, then github, then sourceforge-git  (since gitlab is now the main site).
And probably also mention info that there are some pre-packed downloads.

Unless a page is explicitly about download zip/tar, it should probably first focus on "git clone" then on "packed download".
Title: Re: FPC & Lazarus moving to gitlab
Post by: Fred vS on August 01, 2021, 05:40:16 pm
Unless a page is explicitly about download zip/tar, it should probably first focus on "git clone" then on "packed download".

Of course for lazarus-developers  "git clone" is needed because it will download also all the .git data.

But for Lazarus-users who dont need all the .git data, download zip/tar (that does not include all .git data) is a good/lighter solution, with only one click.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on August 01, 2021, 05:53:48 pm
Unless a page is explicitly about download zip/tar, it should probably first focus on "git clone" then on "packed download".

Of course for lazarus-developers  "git clone" is needed because it will download also all the .git data.

But for Lazarus-users who dont need all the .git data, download zip/tar (that does not include all .git data) is a good/lighter solution, with only one click.
I am not saying it shouldn't be there.
It depends on what the page targets.

Also keep in mind, that anyone who does not only download zips for releases (which are avail on sourceforge too), will be better off with a git clone.
Once you got the clone, you only have to download the differences to what you already have.

The Lazarus fixes zip is currently at 50MB. A full clone iirc at 170MB. Less than 4 times the amount. Once you download your 4th zip....

And if needs must, but I would not recommend it to a git newcomer, there is "--depth".




Title: Re: FPC & Lazarus moving to gitlab
Post by: Fred vS on August 01, 2021, 06:26:45 pm
It depends on what the page targets.

Also keep in mind, that anyone who does not only download zips for releases (which are avail on sourceforge too), will be better off with a git clone.
Once you got the clone, you only have to download the differences to what you already have.

The Lazarus fixes zip is currently at 50MB. A full clone iirc at 170MB. Less than 4 times the amount. Once you download your 4th zip....

And if needs must, but I would not recommend it to a git newcomer, there is "--depth".

Imho a Lazarus newcomer that dont know SVN nor Git will be much more comfortable with Git.
It is a courageous an excellent decision to move to Git.

And congratulation for your work Martin, the conversion of Mantis-bugtraker into Lazarus-GitLab-issues is clear, complete and respectful for patchers.

Fre;D




 
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on August 01, 2021, 06:31:22 pm
And congratulation for your work Martin, the conversion of Mantis-bugtraker into Lazarus-GitLab-issues is clear, complete and respectful for patchers.

The Mantis to GitLab was for the most part Michael. I did help only a little.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Fred vS on August 01, 2021, 06:40:21 pm
And congratulation for your work Martin, the conversion of Mantis-bugtraker into Lazarus-GitLab-issues is clear, complete and respectful for patchers.

The Mantis to GitLab was for the most part Michael. I did help only a little.

Ha, so WoW to both.

(@Michael: like always, impressed by your fast and complete immersion, this time into Git.)
Title: Re: FPC & Lazarus moving to gitlab
Post by: zoltanleo on August 02, 2021, 07:20:07 am
I will also express my words of support.  You are doing a great job.  Please don't stop it.  Even in spite of the indignation of the disaffected.  IMHO, most of the community understands the current difficulties and is patiently waiting.
Title: FPC gitlab update
Post by: trev on August 04, 2021, 10:56:40 am
Quote
04/08/2021 15:55

[T]here is an unforeseen delay in the conversion of the FPC svn -> git repository. I think we're meanwhile at attempt 7...

The conversion takes a lot of time (>48 hours in total). As long as the conversion is not finished, the repository is marked private, and you will get a 404; only team members have access at this point.

Michael Van Canneyt
Title: Re: FPC gitlab update
Post by: El Salvador on August 04, 2021, 11:57:35 am
Quote
04/08/2021 15:55

[T]here is an unforeseen delay in the conversion of the FPC svn -> git repository. I think we're meanwhile at attempt 7...

The conversion takes a lot of time (>48 hours in total). As long as the conversion is not finished, the repository is marked private, and you will get a 404; only team members have access at this point.

Michael Van Canneyt
Thanks for the update!  ;)
Title: Re: FPC & Lazarus moving to gitlab
Post by: prof7bit on August 08, 2021, 08:15:08 am
Its much harder to wait if we can't watch the progress...
Title: Re: FPC & Lazarus moving to gitlab
Post by: trev on August 08, 2021, 11:54:38 am
Quote
Michael Van Canneyt via fpc-pascal wrote on 08/08/2021 19:13

Hello,

After several technical issues (8 tries were needed to convert the FPC
sources to git), the move from svn/mantis to gitlab has been completed.

The FPC sources are now available at

https://gitlab.com/freepascal.org/fpc/source

All FPC git repositories are available at:

https://gitlab.com/freepascal.org/fpc

This is also where bugs can be reported from now on:

All Mantis issues have been converted to Gitlab issues.

Depending on where you report an issue, the Gitlab system will ask you to
select a project under which to report a bug.

Note that e.g. documentation bugs should be reported in the documentation
repository:

https://gitlab.com/freepascal.org/fpc/documentation

etc.

A redirect has been put in place for the old bug system, if possible the
system will redirect you to the correct issue in Gitlab.
A similar redirect will be put in place for the viewvc repository view tool.

The "development" page has been updated with the necessary instructions to
connect with Git:
https://www.freepascal.org/develop.html

More information about git has been placed in the WIKI:

https://wiki.freepascal.org/FPC_git

The lazarus team also placed a version at

https://wiki.freepascal.org/SVN_to_GIT_Cheatsheet

If you see places where we overlooked updating the SVN to git or mantis to
gitlab information, please let us know...

Michael.
Title: Re: FPC & Lazarus moving to gitlab
Post by: MISV on August 14, 2021, 03:37:59 pm
What is the recommended way to update my Lazarus installation on Mac?

In past I would navigate to lazarus folder and use svn up

I need to update sources I think to get access to recent updates for WKWebkit

Title: Re: FPC & Lazarus moving to gitlab
Post by: Leledumbo on August 14, 2021, 10:25:24 pm
What is the recommended way to update my Lazarus installation on Mac?

In past I would navigate to lazarus folder and use svn up

I need to update sources I think to get access to recent updates for WKWebkit
If you don't put any files you never commit, simply delete the old folder then:
Code: [Select]
$ git clone git@gitlab.com:freepascal.org/lazarus/lazarus.git
Don't forget to have a gitlab account and generate SSH keys on your machine if you haven't, then copy the public key by pressing "Add key" on this page (https://gitlab.com/-/profile/keys). Alternatively, you can clone with https but I strongly recommend using SSH keys for convenience.
Title: Re: FPC & Lazarus moving to gitlab
Post by: MarkMLl on August 14, 2021, 10:37:23 pm
Don't forget to have a gitlab account and generate SSH keys on your machine if you haven't, then copy the public key by

That's strange... I don't remember having to sign up like that when using Subversion :-/

MarkMLl
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on August 14, 2021, 10:44:38 pm
In past I would navigate to lazarus folder and use svn up
https://wiki.lazarus.freepascal.org/FAQ_SVN_to_GIT

That's strange... I don't remember having to sign up like that when using Subversion :-/
You don't, if you only download. No username, no password needed.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Leledumbo on August 14, 2021, 11:20:31 pm
You don't, if you only download. No username, no password needed.
I clearly remember my git clone was rejected before registering. Perhaps this only works with HTTPS?
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on August 14, 2021, 11:46:39 pm
You don't, if you only download. No username, no password needed.
I clearly remember my git clone was rejected before registering. Perhaps this only works with HTTPS?
The url you mentioned, looks like an url for an ssh client. And ssh requires you to auth.

If you clone anonymously, you don't need ssh. What benefit would it give you?.

---
Note
    git@gitlab.com:freepascal.org/lazarus/lazarus.git
is not a "git://" url. It is a "user@server:path" for ssh.


EDIT
On the other hand, given that they give you a generic username "git", I wonder if maybe it allows anonymous. I did not test.
Title: Re: FPC & Lazarus moving to gitlab
Post by: trev on August 15, 2021, 12:20:26 am
What is the recommended way to update my Lazarus installation on Mac?
In past I would navigate to lazarus folder and use svn up

See the Wiki - Installing Lazarus on macOS -- Installing non-release versions of the Lazarus_IDE (https://wiki.lazarus.freepascal.org/Installing_Lazarus_on_macOS#Installing_non-release_versions_of_the_Lazarus_IDE) which I updated days ago for git and for svn if you wish to continue using Subversion.
Title: Re: FPC & Lazarus moving to gitlab
Post by: valdir.marcos on August 18, 2021, 05:10:22 am
It depends on what the page targets.

Also keep in mind, that anyone who does not only download zips for releases (which are avail on sourceforge too), will be better off with a git clone.
Once you got the clone, you only have to download the differences to what you already have.

The Lazarus fixes zip is currently at 50MB. A full clone iirc at 170MB. Less than 4 times the amount. Once you download your 4th zip....

And if needs must, but I would not recommend it to a git newcomer, there is "--depth".
Imho a Lazarus newcomer that dont know SVN nor Git will be much more comfortable with Git.
It is a courageous an excellent decision to move to Git.

And congratulation for your work Martin, the conversion of Mantis-bugtraker into Lazarus-GitLab-issues is clear, complete and respectful for patchers.

Fre;D
+1
Title: Re: FPC & Lazarus moving to gitlab
Post by: devEric69 on August 18, 2021, 08:17:58 am
Hello,

I don't come here to be a troll: I perfectly understand the usefulness of GitLab, in order to fork in the cloud, its cost reduction with the associated issues tracker, even to automate centralized group stuff or to create common continuous integration, etc.

I would like to know if some people have already tried some " git2svn " tools to continue to get locally a Lazarus and fpc gitLab's branch inside a local SVN repository (just to receive source code, not to push local changes towards gitLab)?


Sorry (superfluous question): was worrying because i didn't know if fpcUpDeluxe could fetch \ pull Lazarus and fpc from Gitlab. I just saw ther's a Release v2.0.1 of fpcupdeluxe, said as "First wholly functional Gitlab release" (so, I just need to install the Git locally, in order to install and manage these 2 depots locally).

And congratulations for the migration to GitLab.
Title: Re: FPC & Lazarus moving to gitlab
Post by: trev on August 18, 2021, 08:39:29 am
You can still use svn for Lazarus - for example:

Code: Text  [Select][+][-]
  1. svn checkout https://github.com/fpc/Lazarus.git/branches/fixes_2_2 laz_fixes
  2.  
  3. OR
  4.  
  5. svn checkout --depth files https://github.com/fpc/Lazarus/branches all
  6. cd all
  7. svn update --set-depth infinity main

The trunk (aka main) checkout series of commands to to avoid a bug in most Subversion clients.
Title: Re: FPC & Lazarus moving to gitlab
Post by: devEric69 on August 18, 2021, 08:43:27 am
Oh!!? Thanks @trev: finally, i'll continue for a while, testing with your SVN checkout commands.
Title: Re: FPC & Lazarus moving to gitlab
Post by: PascalDragon on August 18, 2021, 08:55:12 am
Oh!!? Thanks @trev: finally, i'll continue for a while, testing with your SVN checkout commands.

Please note however that for bug reports the Git commit hash is a must. We won't take SVN revisions under consideration any longer.
Title: Re: FPC & Lazarus moving to gitlab
Post by: FPK on August 18, 2021, 10:02:58 am
I suggest to use the svn mirror github *only* if there is no working git client for the system one is using. Even for "Read only" users: git is normally faster and a git working copy takes less space.
Title: Re: FPC & Lazarus moving to gitlab
Post by: dbannon on August 18, 2021, 10:56:56 am
And pulling down just a zip or tgz of (eg) trunk source is faster and smaller still. Depends on how frequently you do it ....

Davo 
Title: Re: FPC & Lazarus moving to gitlab
Post by: marcov on August 18, 2021, 12:26:51 pm
I suggest to use the svn mirror github *only* if there is no working git client for the system one is using. Even for "Read only" users: git is normally faster and a git working copy takes less space.

Maybe on *nix, but on Windows, git is much slower IMHO.  Using one repo for both branches is pointless, since I also have multiple lazaruses pointing towards them that use them for debugging etc. (quite often: fixes for work, and trunk to quickly check bugreports).

The SVN checkout was 300MB, a git one is 500MB. Yes, in theory that contains all branches, but IMHO that is of limited use for the source repo.   

Some of the speed differences might be cloud vs own server though, not git vs svn
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on August 18, 2021, 12:42:32 pm
I suggest to use the svn mirror github *only* if there is no working git client for the system one is using. Even for "Read only" users: git is normally faster and a git working copy takes less space.

Maybe on *nix, but on Windows, git is much slower IMHO.  Using one repo for both branches is pointless, since I also have multiple lazaruses pointing towards them that use them for debugging etc. (quite often: fixes for work, and trunk to quickly check bugreports).

The SVN checkout was 300MB, a git one is 500MB. Yes, in theory that contains all branches, but IMHO that is of limited use for the source repo.   

Some of the speed differences might be cloud vs own server though, not git vs svn

One repro, multiple folders:

git worktree add <path>

The 2nd path has its own checkout, but does not need a .git dir.

If you want both on the same remote branch, you need 2 local tracking branches, as you can not switch to the same local branch in both. But otherwise all you need.

(If you have "submodules" read up the doc, and make sure you have the latest version of git)


The speed depends on the cpu.

git checks some of the integrity of the download. svn does not.
So git needs extra cpu time after the download. usually this is very little, unless you are way behind the server and update a massive diff.

Title: Re: FPC & Lazarus moving to gitlab
Post by: FPK on August 18, 2021, 12:47:20 pm
I suggest to use the svn mirror github *only* if there is no working git client for the system one is using. Even for "Read only" users: git is normally faster and a git working copy takes less space.

Maybe on *nix, but on Windows, git is much slower IMHO. 

Initially clone might be comparable slow but all other operations are at least on par with svn according to my experience.

Quote
Using one repo for both branches is pointless, since I also have multiple lazaruses pointing towards them that use them for debugging etc. (quite often: fixes for work, and trunk to quickly check bugreports).

Maybe one repo is pointless for subversion but not for git: with git, you can use so called worktrees to have multiple working copies bound to one local git repository:

cd fpc
git worktree add ../fixes_3_2
cd ../fixes_3_2
git checkout fixes_3_2
du .git
and you will get a few kB for .git

Martin was faster  ::)

Title: Re: FPC & Lazarus moving to gitlab
Post by: marcov on August 18, 2021, 12:52:41 pm
Martin was faster  ::)

I already knew, but the docs say:

Quote
BUGS

Multiple checkout in general is still experimental, and the support for submodules is incomplete. It is NOT recommended to make multiple checkouts of a superproject.


Still I'll experiment with it on secondary machine.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on August 18, 2021, 01:01:59 pm
I have been using it for Lazarus (for over a year, on my private git mirror). And that has worked well.

But yes, I have no idea how it works with submodules.
Title: Re: FPC & Lazarus moving to gitlab
Post by: FPK on August 18, 2021, 01:13:43 pm
Martin was faster  ::)

I already knew, but the docs say:

Quote
BUGS

Multiple checkout in general is still experimental, and the support for submodules is incomplete. It is NOT recommended to make multiple checkouts of a superproject.


Still I'll experiment with it on secondary machine.

Personally I used it only on a RPI with a 8 GB SD and it worked. But in general, I do not consider it worth the effort for FPC or Lazarus.
Title: Re: FPC & Lazarus moving to gitlab
Post by: FPK on August 18, 2021, 08:26:21 pm
I suggest to use the svn mirror github *only* if there is no working git client for the system one is using. Even for "Read only" users: git is normally faster and a git working copy takes less space.

Maybe on *nix, but on Windows, git is much slower IMHO.

Just tested (Windows 10 Pro, NVM SSD), so it's really your IMHO:

c:\fpc\tmp>fptime svn co -q https://github.com/fpc/FPCSource/trunk
89347.000 ms

c:\fpc\tmp>fptime git clone https://github.com/fpc/FPCSource.git
Cloning into 'FPCSource'...
remote: Enumerating objects: 665421, done.
remote: Counting objects: 100% (2234/2234), done.
remote: Compressing objects: 100% (706/706), done.
remote: Total 665421 (delta 1688), reused 2035 (delta 1524), pack-reused 663187
Receiving objects: 100% (665421/665421), 166.06 MiB | 5.22 MiB/s, done.
Resolving deltas: 100% (535772/535772), done.
Updating files: 100% (20103/20103), done.
51129.000 ms


Title: Re: FPC & Lazarus moving to gitlab
Post by: prof7bit on September 02, 2021, 10:57:45 am
Creating new issues is disabled. A few days ago it was enabled, then suddenly it was disabled, then yesterday enabled again for a short time and now it is disabled again. This also affects the Lazarus project.
See screensot.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on September 02, 2021, 11:08:39 am
Creating new issues is disabled. A few days ago it was enabled, then suddenly it was disabled, then yesterday enabled again for a short time and now it is disabled again. This also affects the Lazarus project.
See screensot.

You are in the "FPC > FPC" Group.
You need to be in the "FPC > FPC > Source" repro
Title: Re: FPC & Lazarus moving to gitlab
Post by: prof7bit on September 02, 2021, 11:37:05 am
Wow, this is really bad UI. The issue list should be empty if I am in an area where there can't be any issues. And if it instead shows an aggregated list of all sub-project issues there should (for reasons of symmetry) also be a new-issue-button that allows me to choose for which sub-project I want to create a new issue. Or at least a greyed out new-issue button with a hint that no project is currently selected.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Fred vS on September 02, 2021, 11:38:49 am
Hello Martin and Michael.

I am still very impressed by your conversion to gitlab.

I just noted that the old links to mantis-bug tracker in old forum-posts are perfectly redirected to the corespondent issue on GitLab.

WoW.

Fre;D
Title: Re: FPC & Lazarus moving to gitlab
Post by: PascalDragon on September 02, 2021, 01:44:26 pm
Wow, this is really bad UI. The issue list should be empty if I am in an area where there can't be any issues. And if it instead shows an aggregated list of all sub-project issues there should (for reasons of symmetry) also be a new-issue-button that allows me to choose for which sub-project I want to create a new issue. Or at least a greyed out new-issue button with a hint that no project is currently selected.

Well, you'll have to complain to the GitLab developers about that.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Fred vS on September 02, 2021, 02:08:01 pm
Wow, this is really bad UI. The issue list should be empty if I am in an area where there can't be any issues. And if it instead shows an aggregated list of all sub-project issues there should (for reasons of symmetry) also be a new-issue-button that allows me to choose for which sub-project I want to create a new issue. Or at least a greyed out new-issue button with a hint that no project is currently selected.

Well, you'll have to complain to the GitLab developers about that.

And propose a patch to the GitLab developers (GitLab site is open-source).
Title: Re: FPC & Lazarus moving to gitlab
Post by: creaothceann on October 14, 2021, 07:26:09 am
https://gitlab.com/freepascal.org/fpc/documentation says "See https://docs.freepascal.org/ for the rendered online version of the documentation", but the link doesn't load for me...? In fact https://freepascal.org / https://www.freepascal.org don't load either.
Title: Re: FPC & Lazarus moving to gitlab
Post by: trev on October 14, 2021, 08:17:07 am
Weird. The FPC site docs load fine for me. I ended up at https://www.freepascal.org/docs.var
Title: Re: FPC & Lazarus moving to gitlab
Post by: PascalDragon on October 14, 2021, 08:58:19 am
https://gitlab.com/freepascal.org/fpc/documentation says "See https://docs.freepascal.org/ for the rendered online version of the documentation", but the link doesn't load for me...? In fact https://freepascal.org / https://www.freepascal.org don't load either.

There were some hoster related problems yesterday. It should work again.
Title: Re: FPC & Lazarus moving to gitlab
Post by: creaothceann on October 15, 2021, 12:45:39 pm
Very strange... it seems to load properly at home, but not at my work PC.
Title: Re: FPC & Lazarus moving to gitlab
Post by: BSaidus on October 20, 2021, 07:22:52 pm
Helloooooooo,
Tel me gays ! Is there any way to do an SVN checkout to the Gitlab freepascal/lazarus repos ??
Title: Re: FPC & Lazarus moving to gitlab
Post by: JuhaManninen on October 20, 2021, 08:51:03 pm
Is there any way to do an SVN checkout to the Gitlab freepascal/lazarus repos ??
SVN checkout from the Gitlab repository? No, but git clone is super easy.
I see you use Windows. Please install TortoiseGit. It recommends a Git installation URL during the process if you don't have it installed already.
Git installation alone is sufficient, too. It comes with very nice GUI tools "git gui" and gitk.
If you work on FreeBSD, git should work very well. I use it on Linux without any extra GUI tools. "git gui" and gitk are enough for my purposes.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on October 21, 2021, 03:32:20 pm
Helloooooooo,
Tel me gays ! Is there any way to do an SVN checkout to the Gitlab freepascal/lazarus repos ??

Answer has been given before. => Yes: GitHUB provides svn read only access.
(See earlier posts or check GitHUBs docs)

This feature only works, as long as GitHUB provides it. It works to whatever GitHUB supports. There is zero support from our side. And svn revisions can NOT be used in git reports and will be ignored.
Title: Re: FPC & Lazarus moving to gitlab
Post by: Fred vS on October 21, 2021, 03:41:21 pm
Helloooooooo,
Tel me gays ! Is there any way to do an SVN checkout to the Gitlab freepascal/lazarus repos ??

https://stackoverflow.com/questions/18900774/equivalent-of-svn-checkout-for-git
Title: Re: FPC & Lazarus moving to gitlab
Post by: Martin_fr on October 21, 2021, 04:24:44 pm
Helloooooooo,
Tel me gays ! Is there any way to do an SVN checkout to the Gitlab freepascal/lazarus repos ??

https://stackoverflow.com/questions/18900774/equivalent-of-svn-checkout-for-git
We have 2 "svn 2 git translations" on our own wiki:
https://wiki.lazarus.freepascal.org/FPC_git
https://wiki.lazarus.freepascal.org/SVN_to_GIT_Cheatsheet

They are both separate complete mappings. The diff is explained on top of the 2nd one.
Title: Re: FPC & Lazarus moving to gitlab
Post by: BSaidus on October 21, 2021, 08:43:15 pm
Thanks guys, I see that "GITLAB" do not give svn access to repos, unlike "GITHUB".
Anyway, I use TurtoiseGit for access to GitLab freepascal/lazarus repositories.

Thank you.
TinyPortal © 2005-2018