Lazarus

Miscellaneous => Jobs => Topic started by: lgrfbs on September 27, 2020, 04:40:45 am

Title: Purchase of service: Read and save data from piko measuring cart.
Post by: lgrfbs on September 27, 2020, 04:40:45 am
To those of you reading this, I posted in another thread but that thread went in a different direction, so I try in my own thread.

I am in need of someone who can write together a program described below.
I am in Sweden and am NOT a company, but am willing to pay for the job (within reasonable levels that both can live with)

What do you consider to be the minimum amount for a small job that has the following parts: WiFi, reads data and saves to CSV file?
Please specify which version of my idea (1 to 3) you can build and what it will cost.

--- More info ---------------------------------------------
Some time ago I went and bought PIKO's measuring cart which in itself is a nice cart but the software for Windows has some bugs that PIKO has not done anything about, to get rid of the errors.
The main problem with PIKO's software is that the radius of the curves is not saved in the CVS file.
Values for the radius of the curves are displayed live in the program, so I know that the cart transmits the measure value and that the program receives the data, but the CVS save function does not include the real measurement values for the radius of the curves, but it will only be 0 at that position in the CVS file.
PIKO's measuring cart homepage: https://www.piko-shop.de/?a=mw (https://www.piko-shop.de/?a=mw)

This is a model railway wagon in scale HO (1:87).
What is measured is:
* DCC data
* DCC Volts
* Track inclined width - right/left (dosed curve)
* Track inclined length - up/down (hill)
* Track Radius
* Mileage
* Speed

 --*---

This is what I want the program to do:
1 - Basic function:
* Connects to the measuring cart's WiFi AP
* Reads streamed data from the measuring cart.
* Saves the received raw data as a CSV file locally on the computer/drive.
* Target OS Windows, MaC & Linux
* Open source and publicly downloadable.

2 - Luxury version 1:
* The received data is decoded to human readable values before being saved in the CSV file.
* The program has a GUI that presents received data live in some neat way.

3 - Giant luxury version:
* All measured data is saved as CSV and STEP/STL (3D) file.
* GUI has a 3D window where measuring data is presented as 3D graphics.
* A function to be able to virtually "ride around" on the virtual model railway that collected data has created.


Some row data from Wireshark:
http://lgrfbs.org/SEF/MW20190216.zip (http://lgrfbs.org/SEF/MW20190216.zip) (140 Mbyte)
Title: Re: Purchase of service: Read and save data from piko measuring cart.
Post by: MarkMLl on September 27, 2020, 10:55:53 am
Looks fun, but I suggest that the place to start is asking them if they'll share their source with you, and if they decline a very careful examination of the binary to see if in actual fact it includes anything which is GPLed: they have download links for .pfw files and from elsewhere "The pfw file is basically a filesystem for an embedded linux system. It is in the ext3 format, and can be mounted on linux."

Does the wagon really contain a WiFi access point? Decoding that data looks non-trivial, and you'd be far better finding a community of people with the same interest to attack it.

MarkMLl





https://pacemakerdevice.org/forum/viewtopic.php?t=165
Title: Re: Purchase of service: Read and save data from piko measuring cart.
Post by: lgrfbs on September 27, 2020, 05:59:11 pm
PIKO does not normally respond to technical suggestions or complaints, I have informed them that there is errors in the Windows program.
They have simply considered (my guess) it is NOT worth answering me regarding questions like if there is an API or what license the software is under and this has been going on type since I bought the cart.

Presumably PIKO has purchased the service "create windows programs for us" and that the contract on this service has ended and PIKO does not have or owns the program code themselves.
Have researched a bit and the Windows program is written in Visual Basic.

"Does the wagon really contain a WiFi access point?"
Yes, it does - a small one with a maximum of 4 connections and it can even connect to your local router and thus support more clients as I understood it.
Title: Re: Purchase of service: Read and save data from piko measuring cart.
Post by: MarkMLl on September 27, 2020, 06:11:28 pm
It doesn't look particularly difficult to duplicate. You'd need something like an ESP32 board, an OLED display, and a small handful of sensors (voltage, accelerometer, optical sensor looking at a reflective patch on a wheel to count chains and furlongs or whatever your equivalent is :-)

MarkMLl
Title: Re: Purchase of service: Read and save data from piko measuring cart.
Post by: lgrfbs on September 28, 2020, 07:46:09 am
You are absolutely right, with a little time you can build your own measuring trolley that has the same or better specifications than the PIKO wagon.

Now it is that I'm not very interested in building my own measuring cart right now, the problem with PIKO's Windows program is that once you save your run , so it will be only zeros for curve radius and when you load the CSV file, the diagram for the radius of the curves is just a straight line which means that the railway track never turns in any direction.
So I have some choices:
A; I find someone who is awesome at editing windows EXE files and just goes into the file and fixes what's wrong with PIKOS's program.
B; I get someone to build me a new listening program that fits the measuring cart.
Title: Re: Purchase of service: Read and save data from piko measuring cart.
Post by: trev on September 28, 2020, 08:08:46 am
I doubt hex editing the executable will work. I see there is an Android version which will no doubt reveal the source though...
Title: Re: Purchase of service: Read and save data from piko measuring cart.
Post by: lgrfbs on September 30, 2020, 07:08:41 pm
Really interesting trev, how do you get the source code out of Android software then?

MarkMLl, I've looked at the link you added to your post, but don't understand what to get out of it.

I appreciate any suggestions, I'd rather just get the PIKO software to work or fix a replacement that does almost the same job.

If we look at Option 1, what would it cost me?

The reason I didn't put any price idea in the first post of this thread is because I didn't want to offend anyone with that way level too low.

Another option is that I keep my money and install a VM and run decompulerning on the exe file and get out visual basic code.
Fixes the error and compiles a new exe file.
Have absolutely no idea if it will work as well in reality as it looks on Youtube.

Another option is that I sit down and write the program myself with the forum / internet as support, but then we are there again with having time for it too.
Title: Re: Purchase of service: Read and save data from piko measuring cart.
Post by: MarkMLl on September 30, 2020, 07:24:42 pm
Really interesting trev, how do you get the source code out of Android software then?

MarkMLl, I've looked at the link you added to your post, but don't understand what to get out of it.

I appreciate any suggestions, I'd rather just get the PIKO software to work or fix a replacement that does almost the same job.

If I understand that link correctly, you'll get a decompressed Linux filesystem that you can mount and inspect. There's a reasonable- not overwhelming but reasonable- chance that the authors have written that end in something like Python, which you will be able to inspect. Or alternatively, that you will be able to prove that they're abusing the GPL, in which case particularly if you could get a few more users together you could probably put pressure on them to make source available.

Attacking that end will probably be mechanical, i.e. as described in that link, and as such rather easier than decompiling VB.

MarkMLl
Title: Re: Purchase of service: Read and save data from piko measuring cart.
Post by: lgrfbs on October 01, 2020, 09:56:01 am
Thus these pfw files are for updating software in the wagon, so then should IF I read and understood correctly there is a small Linux computer or resembled in the wagon?!
Title: Re: Purchase of service: Read and save data from piko measuring cart.
Post by: MarkMLl on October 01, 2020, 10:15:38 am
Thus these pfw files are for updating software in the wagon, so then should IF I read and understood correctly there is a small Linux computer or resembled in the wagon?!

Probably, or at least- if that link describing the format is relevant and correct- something that uses a unix-style filesystem to load the processor. And that might expose useful information.

Now I'm not saying it's a sure-fire solution, but it's worth investigating before you wade into trying a decompilation... if nothing else so that you can finger the manufacturer for abusing the GPL.

MarkMLl
Title: Re: Purchase of service: Read and save data from piko measuring cart.
Post by: lgrfbs on October 01, 2020, 10:48:29 am
The manufacturer PIKO have not written that the license for the program is GPL
https://www.piko-shop.de/index.php?vw_type=5&vw_id=94&vw_name=detail
Quote from: Text from PIKO

** Licensing agreement for the usage of PIKO software
This licensing agreement is a legally binding agreement between you and the PIKO Spielwaren GmbH.
 It concerns software, that can be downloaded from this website.
 By downloading or using this software, you agree to these licensing terms. If you do not agree to these licensing terms, you do not have any rights to the software and you have to immediately delete any parts of the software you have already downloaded

1. Licensing The PIKO Spielwaren GmbH grants you the non-transferable, limited and non-exclusive right to use the software only for your personal purposes. You are not allowed to load the software on a RAM unit, on a hard drive, or install it on any other data carrier of a computer, a hardware product or a mobile device. By using the software, you do not gain any ownership rights, copyrights or any other rights of the software. The content of the software is licensed and not sold.
 
 2. Illegal usage
 Neither you nor any third party directed by you may license, copy, reproduce, transfer, lend, sell, lease or distribute the software to third parties or commercially offer it on a pay-for-play basis for a fee or any kind of payment.

 3. Copyright
 You acknowledge that all copyrights of the software are reserved by and remain at the PIKO Spielwaren GmbH. You will not interfere with or diminish these rights, neither through action or omission. You can be held responsible for any violation of copyright by the PIKO Spielwaren GmbH.

 4. No guaranties
 The PIKO Spielwaren GmbH offers the software “as seen” and excludes any guaranties, as long as it is legal. Especially, we exclude any guaranties for the quality of the content or its purpose of utilisation. The PIKO Spielwaren GmbH does not guaranty that the software works without interruptions, errors or viruses nor that it meets your special requirements. It is possible that rights differ, depending on the legislation.
 
5. Disclaimer
 The PIKO Spielwaren GmbH is not liable to prosecution for any damages that can directly or indirectly be associated with the software or parts any part of it. The PIKO Spielwaren GmbH can only be made responsible in cases, where it cannot be excluded by law. Under no circumstances it is possible to make the PIKO Spielwaren GmbH reliable for damages to business, data, profits, assets or for any other kinds of indirect economic damages.
 
 6. General information
 You are not allowed to pass on any rights you have been granted with this license to third parties.
 The PIKO Spielwaren GmbH is not obligated to offer you any warranty, technical support or upgrades for the software.
 
 You agree to comply to all applicable import and export regulations and that you are solely responsible to obtain all necessary licenses for the export, re-export, transfer or import of the content.
 In the case, that this licensing agreement or parts of this agreement turn out to be illegal or unenforceable, all other parts of the agreement remain untouched and legally binding.
Title: Re: Purchase of service: Read and save data from piko measuring cart.
Post by: MarkMLl on October 01, 2020, 11:08:51 am
Exactly, and there was really no need to C&P that boilerplate since I (and anybody else relevant here) have already read it. But if they've used GPL-licensed components in their software you can lean on them.

MarkMLl
Title: Re: Purchase of service: Read and save data from piko measuring cart.
Post by: lgrfbs on October 13, 2020, 06:29:47 pm
Sorry I copied the entire license from PIKO's website, the idea was just to archive what it looked like just when we discussed this.

Have run Wireshark against the cart today and it runs UDP on port 55555, I have recorded screen when I did the test, the movie was 1:07 hour long, so if anyone wants to watch please let me know from you so I post it on my web server.

MarkMLl, I have a LinuxMint box, how should I do to mount PFW files, as you suggested.
Namely, can not find anything online (bad keywords maybe?) that describes the approach.
Title: Re: Purchase of service: Read and save data from piko measuring cart.
Post by: MarkMLl on October 13, 2020, 06:38:43 pm
I gave you a link in #1 in this thread. Work through it as far as you can, then start asking questions... or ask questions if what you see appears to differ grossly from what's being discussed there.

MarkMLl
Title: Re: Purchase of service: Read and save data from piko measuring cart.
Post by: lgrfbs on October 16, 2020, 08:53:50 am
Problem the commando:
Quote from: https://pacemakerdevice.org/forum/viewtopic.php?t=165
unzip -d ./extracted_dfw_1412 dfw_14142.pfw

do nothing at all, I thinking this file is not a Pacemaker firmware file.
Screencast:
http://hobbywebb.se/Q/Kazam_screencast_00000.mp4 (download)
Title: Re: Purchase of service: Read and save data from piko measuring cart.
Post by: dbannon on October 16, 2020, 09:34:14 am
unzip -d ./extracted_dfw_1412 dfw_14142.pfw

No, the file you download is a zip file, it contains the pwf file.

The file command says the pwf file is data, strings command shows nothing recognizable.  I'd suggest its encrypted or encoded some other way and requires some specific application to make use of it.

I would line up with Mark, it would be easier to build a new system using the existing rolling stock than to reverse engineer. While agree its possible it contains GPL software, you cannot establish that with out breaking terms of the (possibly but only possibly invalid) license.

Davo
Title: Re: Purchase of service: Read and save data from piko measuring cart.
Post by: MarkMLl on October 16, 2020, 09:49:11 am

$ mkdir MesswagenComplete_v1_06_00
$ cd $_
$ unzip ../MesswagenComplete_v1_06_00.zip
Archive:  ../MesswagenComplete_v1_06_00.zip
  inflating: MesswagenComplete_v1_06_00.pfw 
$ ls
MesswagenComplete_v1_06_00.pfw
$ file *
MesswagenComplete_v1_06_00.pfw: data


Right, so the .pfw file is data. but the instructions you are being given clearly say

"If it is zipped, simply..." and it's not zipped.

However at that point the filesize is not what that site suggests it should be, and it doesn't mount as an ext3. xxd shows that it starts


00000000: c6c4 83c9 79eb 9944 2ea3 212c 7736 679c  ....y..D..!,w6g.
00000010: 72dd b90a 6dac 5050 0884 2ad0 6f3f 29ec  r...m.PP..*.o?).
...


which doesn't match the description of cramsfs at https://www.kernel.org/doc/Documentation/filesystems/cramfs.txt but it could still possibly be some variant... or been obfuscated to make this kind of inspection difficult.

I'm afraid that that's as far as I'm prepared to go. I still feel it might be worth your while poking at that firmware image a bit more, but otherwise you're very much on your own (form the POV of this forum) if you start trying to unpick a compiled VB or whatever program to run on the host PC.

MarkMLl
Title: Re: Purchase of service: Read and save data from piko measuring cart.
Post by: MarkMLl on October 16, 2020, 09:54:30 am
I would line up with Mark, it would be easier to build a new system using the existing rolling stock than to reverse engineer. While agree its possible it contains GPL software, you cannot establish that with out breaking terms of the (possibly but only possibly invalid) license.

There's a single block of ffff in that image which suggests it's not compressed/encrypted, and the location of that relative to the end of the image is interesting. Don't think OP's told us what kind of chip it is... the bottom doesn't look like jump vectors but the loader might not be putting it at 0x0000, which complicates the usual "the bottom of Flash contains a recognisable jump opcode so the chip is a xxxx" trick.

I think that's about as much as can be done.

MarkMLl
Title: Re: Purchase of service: Read and save data from piko measuring cart.
Post by: lgrfbs on October 16, 2020, 11:09:45 am
CPU1: Atmel XM32E5-U 1714B  PH A93QQA - This is on the LCD board
CPU2: XMEGA 32A4U MH1710 KOREA-AB 35965E A88P8A  - Main PCB, This is on the key and USB board
FLASH1: adaslot1711 45DB641E SHN

Sure, I could build and program my own measuring trolley, but now I want to get to this so that other model railway hobby practitioners can also get much more out of this wagon as it is already on the market.
I have got the feeling that PIKO is done with this product and will not update or correct the program to Windows and let alone give Mac/Linux users a program for this cart.

How hard would it be to write a program that listens to the raw data from the cart?
What I need to tell the program is the IP number that the cart has for the rest is known from WireShark:
UDP on port 55555.

I am deeply grateful for all the answers I have received so far, THANK YOU.

All photos:
http://hobbywebb.se/Q/PIKO_MW.zip (142Mb) 

//edit:
//fixed images link.
Title: Re: Purchase of service: Read and save data from piko measuring cart.
Post by: lgrfbs on October 16, 2020, 11:31:13 am
Don't think OP's told us what kind of chip it is...
I couldn't tell you before which CPU was used in the cart.
Because the wagon was not opened until today to read the text on the circuit parts and take some photos.
Title: Re: Purchase of service: Read and save data from piko measuring cart.
Post by: MarkMLl on October 16, 2020, 11:50:48 am
CPU2: XMEGA 32A4U MH1710 KOREA-AB 35965E A88P8A  - Main PCB, This is on the key and USB board

OK, that's new info. I'm not going back to check, but I suspect that the link I gave you that discusses decoding the file is for a different target in which case it's clearly not relevant. An ATMEGA disassembler might make some sense of that binary, but the important thing is that it's not something simple like a Python script which is immediately readable.

Quote
Sure, I could build and program my own measuring trolley, but now I want to get to this so that other model railway hobby practitioners can also get much more out of this wagon as it is already on the market.

Which is likely to mean new electronics as a drop-in replacement.

Quote
How hard would it be to write a program that listens to the raw data from the cart?
What I need to tell the program is the IP number that the cart has for the rest is known from WireShark:
UDP on port 55555.

Well, you've got the Wireshark captures. My own recollection from looking at them in the past is not very easy... at the very least I think you're going to need to make a "rolling road" arrangement so that you can move the wagon at a simulated controlled speed, simulate different cambers and so on.

Seriously. Your best bet is going to be to look at how that wagon is put together, and work out whether you could both make a drop-in replacement for existing wagons and apply the same thing to a new design... providing that you use cleanroom code you're probably legally OK. Oh, and for this you probably want to use TCP, unless you're pretty confident that your code is handshaking at a higher level.

Railways have used dynamometer cars since the C19th, and I'd remind you of Babbage's story of helping out when the only way to check the track was to release a wagon and time how long it took to stop (in his case, he arrived at the next station after jury-rigging a sail). There's almost certainly a niche market for somebody who can convert a wagon in a customer's chosen livery into an effective dynamometer, and being in control of the physical side of that would probably be adequate to protect against flagrant reuse of any code you choose to open-source.

Go for it man, and remember that if you don't there's multiple people in here that might see a viable opportunity... and many more reading courtesy of Google.

MarkMLl
Title: Re: Purchase of service: Read and save data from piko measuring cart.
Post by: lgrfbs on October 16, 2020, 12:23:19 pm
The term "cleanroom code" I have not heard before, guess that means I have written everything myself and not looked at anyone else's code.
In the same way as React OS has done.

Babbage's story was new information for me as a Swede, thanks for it was interesting.

Quote from: MarkMLl
"Go for it man, and remember that if you don't there's multiple people in here that might see a viable opportunity... and many more reading courtesy of Google."
So it may be but NO ONE has given me any price proposal at all on what was the start of this thread.
Title: Re: Purchase of service: Read and save data from piko measuring cart.
Post by: MarkMLl on October 16, 2020, 12:37:51 pm
The term "cleanroom code" I have not heard before, guess that means I have written everything myself and not looked at anyone else's code.
In the same way as React OS has done.

In the most rigorous case, there's one team who develops a specification from a program, which possibly includes disassembling it, and a completely distinct team who develops a replacement which is functionally identical. My understanding is that European law allows disassembly in order to develop something which is functionally equivalent, but the ReactOS- and for that matter FPC- approach that people intimately familiar with the Windows/Delphi codebase should not contribute to the open-source one is preferable.

Quote
Babbage's story was new information for me as a Swede, thanks for it was interesting.

He was an interesting man. More interesting, I'm afraid, than Per Georg Scheutz :-)

Quote
So it may be but NO ONE has given me any price proposal at all on what was the start of this thread.

Well, you must admit that the information has dribbled out fairly slowly, and I'm afraid that most of us- for the moment at least- have our own interests and demands on our time.

MarkMLl
Title: Re: Purchase of service: Read and save data from piko measuring cart.
Post by: dbannon on October 17, 2020, 06:56:06 am
.....
So it may be but NO ONE has given me any price proposal at all on what was the start of this thread.

Honestly, I don't think its a commercial prospect. Someone may do it because its fun, they enjoy the challenge or really want the outcome. But if someone was to agree to work at an hourly rate and just do it for the money, you might be looking at 100 hours to make a start, perhaps get to the point where you can tell if its viable or not. What would you accept as an hourly rate ?

A much better approach would be for you to keep working on that wireshark data, if you can find some sort of API, then getting someone to write some code that uses the API would be heaps more practical. Put that research up on (eg) github and see who is also interested.

The data coming from the thing will probably be in some common format, maybe xml, json etc. If you can spot the deliminator, maybe you can start breaking each lump of data down. You may be able to label each field one by one.  very slow process !

Davo

Title: Re: Purchase of service: Read and save data from piko measuring cart.
Post by: trev on October 17, 2020, 07:16:39 am
So it may be but NO ONE has given me any price proposal at all on what was the start of this thread.

Without access to the hardware, it's not going to be something someone knocks up in a week. If you're not wanting to do it, it would be best to tap into the community of users of the hardware and see if anyone else wants to (a) help (b) fund it.
Title: Re: Purchase of service: Read and save data from piko measuring cart.
Post by: lgrfbs on October 17, 2020, 08:12:18 am
Thanks for an honest estimate of the time, when I'm not a professional programmer so I don't know what a reasonable hourly/total cost of the "1- Basic function" program would be.

Thank you for helping me to see that the only economic thing is that I myself dive into the internet's waves to find code segments so that I can write the "1 - Basic function" program.

A few of the reasons I added the thread on this forum were that I like Lazarus as well as that I know there are model railers here too.
Thought it was a double win type.

I know the first post may feel poor on information, but that's what I knew then and when this thread took some momentum I "had to" look for that information, so you (the reader) and I have learned more about this cart.

So I withdraw my offer to pay someone to create a program for me.
TinyPortal © 2005-2018