pcb-rnd knowledge pool
How pcb-rnd does not lock in users
| format_lockin by Tibor 'Igor2' Palinkas on 2021-07-14 | Tags: insight, geda, file format, file, format, data model, data, model, fork |
Abstract:
During an annual gschem file format flamewar on the geda-user mailing list,
Bert (geda/pcb developer) tried to steer things in a specific direction. For
this he cited an example from pcb-rnd. He claimed that the fact that
pcb-rnd invented a new native file format somehow blocks users from
moving between pcb-rnd and geda/pcb or using them in parallel.
Of course this claim has no base at all. In response I wrote the
following open letter to Bert, in which I tried to explain, backed up with
facts and data, why he is wrong in his claim about pcb-rnd file format and
why he is wrong in what geda should learn from this.
At the end I also tried to present a conclusion what I think geda should
really learn from pcb-rnd.
Hi Bert,
Sorry, this is real long, but... I recommend you read it, even if you will hate what you see, because this insight may be useful for you with your project, pcb.
This was brought to my attention:
https://www.delorie.com/archives/browse.cgi?p=geda-user/2021/07/14/05:57:25
more specifically this paragraph:
"pcb has spawned a clone (2014-ish) with an entirely different native file format, I won't call that a "vendor lock" in though, it's effectively preventing users to swap applications "on the fly", and we can learn from this."
This claim is ignoring facts and is false. If you weren't who you are within geda and pcb, I'd say this is just an innocent mistake: you just follow the best geda traditions of making bold (and false) claims without doing even a 2 minute research beforehand. But in your position, I am sorry, but I have to assume malice. Or in short: you lied.
I am no longer subscribed to geda-user - in which the geda culture which is based mainly around bold-claim-without-any-base and straight-out-lie played a big role so I am not going to post my reply there. I will publish it as an open letter in pcb-rnd knowledge pool instead. (Btw, that's the main reason I bother to write it in the first place.)
These little FUD games also play a big role in: why there's zero cooperation within geda and why the whole geda project is just fading away.
What you said above is simply false. Unlike you, I do have facts to back up my claims.
I am going to show you 6 facts about there's less file format "lock-in" on pcb-rnd side than on pcb side, and how/why you are losing users instead. You can verify them. Some are cheap to verify (looking at releases and changelogs and running pcb-rnd for once). Some are expensive to verify: you'd need to do what I often do: talk to users and make polls.
I will also draw some conclusions about where claims like yours and the overall strategy you do (both as Bert of pcb and as the geda project in generall) actually leads. If you really care about what happens to gschem and lepton, these are the things you should think over and talk over with them, not false file format claims.
Fact 1: pcb-rnd to pcb flow has no file format "lock in" of _any_ sort (easy to verify)
Pcb-rnd does have a new native file format but it also does support geda/pcb's file format. It always did. During the transition of a whole data model rewrite (which took much more than a year), there was exactly zero releases that did not fully support the legacy geda/pcb format. In fact even the development (svn HEAD) version didn't break geda/pcb file format support for longer than a few days of that whole more-than-a-year period!
This does include board files, footprints and the legacy netlist formats.
Even more, in the footprint library, you can place footprints in the geda/pcb .fp format and as a user you won't notice any difference on schematics import, library window or footprint related actions.
Pcb-rnd can load _and_ save the legacy geda/pcb file formats - both board and element files.
So saying that pcb-rnd's file format is a lock-in or burden in switching between pcb and pcb-rnd in any direction is simple disinformation.
Fact 2: pcb-rnd is more compatible with pcb than pcb with pcb! (easy to verify)
Actually pcb-rnd does all the above _better_ than pcb itself. We do save in 3 different geda/pcb file format flavors (in what kind of units we use). Plus we do load all known geda/pcb board and fp file format versions. We partially load even some real old pcb files that pcb 4.3.0 chokes on! Example:
http://repo.hu/projects/pcb-history/releases/pcb-history-1.1.tar.gz
see file called "demo", which is the example board distributed with pcb around that time.
Note: pcb-rnd's load of this ancient file format is not perfect, there's a problem with the layer stackup, and an error with one of the components - but hey, at least you get traces and pins, and it's not an empty board like in pcb!
Note: this fact has very little practical impact, but shows well that pcb-rnd is exactly the opposite of "moving forward breaking compatibility", which you suggested.
Fact 3: if any of the two project has file format lock-in, that's pcb, not pcb-rnd (easy to verify)
pcb-rnd reads board files of geda/pcb, kicad (2 different file formats: s-expr and the legacy line based), eagle (2 formats: xml and the legacy binary), pads ascii, specctra dsn, hyperlynx, autotrax and the exchange format tEDAx.
pcb-rnd reads footprints of most of the above formats plus BXL and mentor cell.
pcb-rnd imports from more than 12 different netlist formats.
pcb-rnd can _save_ boards in geda/pcb, kicad s-expr, dsn (in progress, will most probably be ready by the next release, in august), PADS ASCII, autotrax and the exchange format tEDAx.
In the same time, pcb loads and saves only its own board and footprint file format, can import netlist only from geda and tinycad and maybe 1 more format.
So with pcb-rnd you can easily move board files and footprints between a wide range of PCB editors. If you draw something in pcb-rnd, we, pcb-rnd, give you features to save it in the format of many other editors. We, pcb-rnd actively make you able to move away from pcb-rnd, if you wish to.
In the same time if you draw something in geda/pcb, that pretty much remains there forever (unless you use pcb-rnd to convert it...). There are/were some half-steamed scripts for partial conversions, but nothing as complete as any of pcb-rnd's save. And especially not maintained and promoted by pcb developers.
And you say if someone switches to pcb-rnd, there's "effectively" no way to leave it, unlike, uhm, in pcb, and that's "because of the new file format in pcb-rnd"? Really?
Fact 4: there _is_ moving forth and back between pcb and pcb-rnd (you will need to talk to users to verify this)
One of the many things pcb-rnd gets right and pcb failed at in the past decade is contact with users. We have a lot of communication between users and developers. As the lead developer, I do a lot of user support and I do try to understand their use cases and workflows.
In my experience, from about 20 geda/pcb users who _ever_ try to draw or edit anything semi-serious in pcb-rnd, about 19 switches to pcb-rnd and stays. I know many of them one by one, and for some of them I remember why they switched.
But for disproving your false claim, that ~1 user out of the 20 who did not stay or did not fully switch is the interesting one for us, for now.
I know 1 who uses pcb and pcb-rnd in parallel and I know about 2 or 3 who really tried pcb-rnd and after a while switched back to pcb and stopped using pcb-rnd. These cases are empirical proof that switching back is possible.
Fact 5: vast majority of the users who _ever_ try pcb-rnd and decides to communicate with me end up switching to pcb-rnd and remain on pcb-rnd, and it has nothing to do with file formats (verification requires you to contact and interview users)
They stay not because they couldn't save their stuff in .pcb/.fp and go back any time. The real reason for them to stay is what you just don't want to know about: that pcb-rnd did implement tons of features those users really need and pcb did not implement.
This is the real "lock in", not the file format: once an user realizes pcb-rnd gives them what they need for efficient work while pcb gives none, they are "locked in" to pcb-rnd in the sense they simply won't see any value in going back to pcb. This would change only if pcb-rnd is messed up or if pcb reproduces the advances pcb-rnd achieved.
Pcb-rnd did make relevant progress in the past 6 years while pcb did not. And by relevant I mean things users really want and explicitly request, like editable layers (mask, paste), things that other EDA tools supported for decades like keepout, padstacks, anything-goes-in-footprints, advanced search, etc.
I've looked at what you've done in the past 1..2 years in pcb, looked at some NEWS files of yours. From user's perspective: pretty much _nothing_. Users really don't care about your xml export that no software, not even pcb is able to load. They really don't care about doxygen commets or a markdown BOM exports. If you want to see what they want, look back at 2 decades of neglected feature requests on geda-user@ - I did pick up a lot of those and implemented them in pcb-rnd, and trust me, they do yield positive feedback.
The only data model related news I've found, in 4.3.0, is a new centerpoint/length smd pad description. Which pcb can read but can't save in the same form but converts back into the subiptimal and buggy line-with-thickness style description... Now put this next to pcb-rnd's padstacks (which are available for longer than 2 years already!), the ability to have any shape, even different shapes on top and internal copper, real slots, etc.
(You also had a layer syntax upgrade in your format, but I don't see the code is using that for anything, plus I already see how it will cause you problems whenever you will try to start using it - because it was not designed carefully. But at its current form, at the moment, it is just random noise in the file format, nothing relevant to the user)
And you claim it's the file format that keeps pcb-rnd users from switching back to pcb, not the utter lack of progress or the 90's feature set that pcb offers and even kicad is surpassing lately....
Fact 6: you _are_ losing users, but not to pcb-rnd (easy to verify)
Reading between your lines, I think you write such things because you probably think pcb-rnd did bad to pcb and/or geda. I am going to disprove this too, with hard data - at least I will prove that if you assume the biggest damage pcb-rnd could ever do, according to the numbers, that's still a little drop in the sea of reasons why pcb and geda is not going as well as you want it to go. And I will explain some mechanisms why I think _your_ actions, and the generic state of geda in fact contribute the majority of the reasons for geda going downhill.
For this, first you will need to accept that pcb and geda are going dowhill, usage-wise.
Unfortunately there's not too much publicly available statistics on how many users our projects have. The only "big sample" stats I know about is debian's popcon:
this is how many users have the software installed:
https://qa.debian.org/popcon-graph.php?packages=pcb-gtk%2Ckicad%2Cpcb-rnd&show_installed=on&want_legend=on&from_date=2006-01-01&to_date=&hlght_date=&date_fmt=%25Y-%25m&beenhere=1
this is how many users ran it the past 2 weeks:
https://qa.debian.org/popcon-graph.php?packages=pcb-gtk%2Ckicad%2Cpcb-rnd&show_vote=on&want_legend=on&from_date=2006-01-01&to_date=&hlght_date=&date_fmt=%25Y-%25m&beenhere=1
(Note: this data only contains a portion of users on two levels: only a portion of users use debian, and only a portion of debian users will enable popcon reporting. However, with numbers this big, I think we can't assume this portion behaves differently from the whole. Which means: when you see "grows from 50 to 100 in 2 years", it's not the absolute value of 50 and 100 that matters, because there is an unknown multipler from this portion to the whole, but the rates and trends.)
Look at the graphs closely. You will notice that:
- pcb was in par with kicad, both in absolute numbers and in growth between 2008 and 2012. Remember what happened around that time? There were a bursts of development in pcb around 2005, 2009 and 2011. Real big and important things, like introducing HIDs and switching to nanometer. And remember what happened after 2011 to pcb? Pretty much nothing comparable. Then kicad got the CERN sponsoration and developement sky-rocketed - this happened around 2015. Do you think it is a coincidence that pcb started to stagnate in 2012 and start to go downhill in 2014? (for this look at the actual usage graph, not at the 'installed' graph: people tend to keep legacy software installed, just in case, even when they already switched away)
- with your gedaism, I hear you saying "it's pcb-rnd's fault! the fork kills pcb!". But looking at the graph that doesn't seem to be true. Look at the usage graph: pcb-gtk's peak was at about 220 debian users. Kicad had 350 in the same peak and pcb-rnd was not even a thing back then, I had like 2 or 3 users. Now look at today: pcb has 90, kicad has 600, pcb-rnd has about 30. So if you look at pcb-rnd users coming from pcb, that would still only explain 30 of the ~100 users you've lost. In a time frame when kicad gained some 250 users!
- So the sad truth is: you are losing users not to pcb-rnd but to kicad most probably. And if that's true, that means geda is losing those users too. Think it over: if someone switches from pcb to pcb-rnd, he will likely keep his previous sch editor (gschem or lepton typically), especially because pcb-rnd is backward compatible with pcb on them too. But if he switches from pcb to kicad, he also switches from geda to eeschema (kicad's sch editor). Partly because the path from gschem/lepton to kicad is at least bumpy, partly because kicad is pretty much an integrated offer, partly because a significant reason for using kicad is that you can share your design with more people (and sch in geda would be in the way on that one).
To back up this claim, look at this one:
https://qa.debian.org/popcon-graph.php?packages=pcb-gtk%2Cgeda-gschem%2Clepton-eda&show_vote=on&want_legend=on&from_date=2006-01-01&to_date=&hlght_date=&date_fmt=%25Y-%25m&beenhere=1
which is software usage in debian for pcb, gschem and lepton.
You can see whole geda has the trend of peaking around 2012..2014, and then going downhill. So it's not pcb alone, it's geda in whole! And gschem went down from 200ish to 70ish in 8 years (that is minus 130), while lepton-eda had only 50 at its peak. Plus it was already going downhill fast years before the lepton fork. Again, you can't say all gschem users lost simply went to lepton - most of them went away from it all, most probably to kicad.
Conclusion:
So the net result of your activity is this: you had a major contribution at a strategic moment in that pcb remained the main and only pcb editor of geda, back in 2016. You decided to keep pcb alive and make it the main option to geda users. Sometimes even using false claims like the one I quoted in the beginning of this wall-of-text, about (non-existing) file format barriers on pcb-rnd-side.
You perhaps hoped this would intimidate users and keep them on pcb, when you probably knew (you had the chance to see in 2015 and half of 2016 already) that pcb-rnd was/will be progressing like 20x faster than pcb.
Half of your strategy worked, but not the way you think. In reality vast majority of pcb users (and geda users in general) are not subscribed to the mailing lost, won't visit irc and won't try to communicate with developers. (Btw, we have this very same problem in pcb-rnd too.) They are the silent majority of the users. Since pcb is what their package manager says geda is, they will simply use that. So this half worked: since geda didn't do anything to tell those users about pcb-rnd, you didn't immediately loose most of them to a better alternative (pcb-rnd), because they simply didn't know about pcb-rnd.
But the other half, that this keeps them on pcb forever seems to fail, badly. Ok, they won't learn about pcb-rnd, because pcb sort of hides it from them, but pcb can't hide kicad. And kicad, just like pcb-rnd, also had huge progress while geda/pcb was idling the past many years.
Thus what you really achieved with "keeping pcb alive" is more users switching from pcb to kicad, which means switching out from geda too. If you look at the data, debian usage graphs, or try to talk with users who migrate away from your software, you will see this happening.
Doing your little FUD game trying to sell users that pcb-rnd will lock them in "because of file formats", won't help in this at all. On geda-user@ you reach only those few who already know about pcb-rnd but already decided not to switch (they will eventually switch to kicad or something totally different - either when they bump into one of the 10000 pcb limitations too hard, or when pcb finally bitrots away). On that mailing list you simply don't reach that silent (non-subscribed) majority who will slowly migrate away mainly to kicad and in a smaller portion to pcb-rnd.
So, if you have read all this, and accepted the data and facts presented, you owe your users a correction on the mailing list. Don't get me wrong: you don't owe me anything; because of exactly the same data above, to me, geda and pcb are pretty much irrelevant. It's your users who deserve better than this. It's your users who are getting in worse and worse situation becasue of pcb - the longer they stay, the more expensive it will be _for them_ to switch (to either pcb-rnd or to kicad). And not because of file formats, because we keep on loading more EDA formats than most users have ever seen, but because of the huge amount of new features and concepts everything-else-but-pcb have in EDA land (including kicad)!
Regards,
Igor2