[SOLVED] Binary STL file writing code

[m4hunk]

So basically I want to write a code that writes a Binary STL file. I've managed to get it work with the ASCII format of the STL file, then it's safe to assume that the facets data are correct. The problem is that the file, that is created and written, is corrupt. The PRINT 3D software from windows can't open it. However there is another software, netfabb Basic 6.4, that can open the file and it shows that approximately "half" of the data gets corrupt (does not show on the software). Therefore, I'm pretty sure that the problem lies with the writing code.

I am following the Binary format from this site: https://fabbers.com/tech/STL_Format

Code: Pascal  [Select][+][-]

1.  
2.  type
3.   Vetor = record
4.           x, y, z : single;
5.   end;
6.  type
7.   facets = record
8.           normal, ver1, ver2, ver3 : Array of Vetor;
9.   end;
10.  
11. procedure cria_BINARY_STL;
12. var
13.   attribute_byteCount, i : UInt16;
14.   intBuffer            : UInt32;
15.   F                      : Longint;
16.   CharBuffer             : UInt8;
17. begin
18.  
19.   attribute_byteCount:=0;
20.  
21.   F:=FileCreate('BINARY_STL.stl');
22.  
23.   if F=-1 then Halt(1);
24.  
25.   CharBuffer:=0;
26.  
27.   for i:=0 to 79 do
28.   begin
29.     FileWrite(F,CharBuffer,sizeof(CharBuffer));
30.   end;
31.  
32.   intBuffer:=num_tri-1;
33.  
34.   FileWrite(F,intBuffer,sizeof(intBuffer));
35.  
36.   for i:=0 to intBuffer do
37.   begin
38.     FileWrite(F,StlData.normal[i].x,sizeof(StlData.normal[i].x));
39.     FileWrite(F,StlData.normal[i].y,sizeof(StlData.normal[i].y));
40.     FileWrite(F,StlData.normal[i].z,sizeof(StlData.normal[i].z));
41.  
42.     FileWrite(F,StlData.ver1[i].x,sizeof(StlData.ver1[i].x));
43.     FileWrite(F,StlData.ver1[i].y,sizeof(StlData.ver1[i].y));
44.     FileWrite(F,StlData.ver1[i].z,sizeof(StlData.ver1[i].z));
45.  
46.     FileWrite(F,StlData.ver2[i].x,sizeof(StlData.ver2[i].x));
47.     FileWrite(F,StlData.ver2[i].y,sizeof(StlData.ver2[i].y));
48.     FileWrite(F,StlData.ver2[i].z,sizeof(StlData.ver2[i].z));
49.  
50.     FileWrite(F,StlData.ver3[i].x,sizeof(StlData.ver3[i].x));
51.     FileWrite(F,StlData.ver3[i].y,sizeof(StlData.ver3[i].y));
52.     FileWrite(F,StlData.ver3[i].z,sizeof(StlData.ver3[i].z));
53.  
54.     FileWrite(F,attribute_byteCount,sizeof(attribute_byteCount));
55.   end;
56.  
57.   FileClose(F);
58.  
59. end;    

[bytebites]

Add packed?

Code: Pascal  [Select][+][-]

1.  type
2.   Vetor = packed record
3.           x, y, z : single;
4.   end;

and simplify writing

Code: Pascal  [Select][+][-]

1. FileWrite(F,StlData.normal[i],sizeof(Vetor))

[howardpc]

Packed is not relevant, since this is not a file of records he is writing, but sequential variables.
m4hunk, you write a value of intbuffer facets to the file, but then follow this with writing intbuffer+1 facets (0..intbuffer). So it is not surprising the file cannot be read by software expecting to find intbuffer facets.

[ChrisR]

I would strongly suggest you use STL only as a format as last resort. If this is the ONLY format supported by the software or printer you are using. The STL format is fatally flawed, as it does not reuse vertices. An indexed vertex file can be much smaller. Leading to smaller files and better usage of graphics card resources
 http://hacksoflife.blogspot.com/2010/01/to-strip-or-not-to-strip.html
Beyond these benefits, since the vertices are not shared, one either must use per-face shading (leading to a faceted, jagged looking surfaces) or must go through the laborious task of unifying vertices with some arbitrary tolerance before being able to compute per-vertex shading.



Here is the code used by Surfice https://github.com/neurolabusc/surf-ice

Code: Pascal  [Select][+][-]

1. procedure SaveStl(const FileName: string; Faces: TFaces; Vertices: TVertices; vertexRGBA: TVertexRGBA);
2. const
3.   kHdrBytes = 80;
4.   kURL = 'https://en.wikipedia.org/wiki/STL_(file_format)';
5. type
6.   TTri = packed record
7.         normal, v1,v2,v3: TPoint3f;
8.         AttributeByteCount: Uint16
9.      end;
10. var
11.   hdr: array [1..kHdrBytes] of byte;
12.   //faces16: array of Tui16;
13.   i, nFace, nVert: integer;
14.   nFace32: UINT32;
15.   mStream : TMemoryStream;
16.   FileNameStl: string;
17.   t: TTri;
18. begin
19.   if (length(faces) < 1) or (length(vertices) < 3) then begin
20.      showmessage('You need to open a mesh before you can save it');
21.      exit;
22.   end;
23.   if (length(vertices) < 3) or (length(faces) < 1) then exit;   //if not CheckMesh then exit;
24.   if (length(vertexRGBA) > 0) then
25.      showmessage('Warning: STL format does not support vertex colors');
26.   FileNameStl := changeFileExt(FileName, '.stl');
27.   {$IFDEF UNIX}
28.   FileNameStl := ExpandUNCFileNameUTF8(FileNameStl); // ~/tst.pl -> /Users/rorden/home/tst.ply
29.   {$ENDIF}
30.   for i := 1 to kHdrBytes do
31.     hdr[i] := 0;
32.   for i := 1 to length(kURL) do
33.     hdr[i] := ord(kURL[i]);
34.   nFace := length(Faces);
35.   nVert := length(Vertices);
36.   mStream := TMemoryStream.Create;
37.   mStream.Write(hdr,sizeOf(hdr));
38.   nFace32 := nFace;
39.   t.normal.x := 0;
40.   t.normal.y := 0;
41.   t.normal.z := 0;
42.   t.AttributeByteCount := 0;
43.   {$IFDEF ENDIAN_BIG}
44.   nFace32 := swap(nFace);
45.   swapPt(t.normal);
46.   t.AttributeByteCount := swap(t.AttributeByteCount);
47.   {$ENDIF}
48.   mStream.Write(nFace32, 4);
49.   for i := 0 to nFace-1 do begin
50.     t.v1 := ptf(vertices[faces[i].X].X, vertices[faces[i].X].Y, vertices[faces[i].X].Z);
51.     t.v2 := ptf(vertices[faces[i].Y].X, vertices[faces[i].Y].Y, vertices[faces[i].Y].Z);
52.     t.v3 := ptf(vertices[faces[i].Z].X, vertices[faces[i].Z].Y, vertices[faces[i].Z].Z);
53.     {$IFDEF ENDIAN_BIG}}
54.     swapPt(t.v1);
55.     swapPt(t.v2);
56.     swapPt(t.v3);
57.     {$ENDIF}
58.     mStream.Write(t, sizeof(t));
59.   end;
60.   mStream.Position := 0;
61.   FileMode := fmOpenWrite;   //FileMode := fmOpenRead;
62.   mStream.SaveToFile(FileNameStl);
63.   mStream.Free;
64.   FileMode := fmOpenRead;
65. end;
66.  

[ChrisR]

Your code looks pretty fine to me except for the first inter you write, which should be the "number of facets in file", not "the number of facets minus one":

Code: Pascal  [Select][+][-]

1.  
2. ...
3. intBuffer:=num_tri;
4. FileWrite(F,intBuffer,sizeof(intBuffer));
5. for i:=0 to (intBuffer-1) do
6. ...
7.  

[m4hunk]

Add packed?

Although the code looks more readable, the results are the same.

m4hunk, you write a value of intbuffer facets to the file, but then follow this with writing intbuffer+1 facets (0..intbuffer). So it is not surprising the file cannot be read by software expecting to find intbuffer facets.

Your code looks pretty fine to me except for the first inter you write, which should be the "number of facets in file", not "the number of facets minus one":

Thank you howardpc and ChrisR this bug passed without me noticing it. Sadly, the file still becomes corrupt.

I would strongly suggest you use STL only as a format as last resort. If this is the ONLY format supported by the software or printer you are using. The STL format is fatally flawed, as it does not reuse vertices. An indexed vertex file can be much smaller. Leading to smaller files and better usage of graphics card resources
 http://hacksoflife.blogspot.com/2010/01/to-strip-or-not-to-strip.html
Beyond these benefits, since the vertices are not shared, one either must use per-face shading (leading to a faceted, jagged looking surfaces) or must go through the laborious task of unifying vertices with some arbitrary tolerance before being able to compute per-vertex shading.

Thank you ChrisR. I will take a look. However, I really want to make this work because now I am intrigued on why this is not working, it is supposed to be simple. I am having difficulty to debug because of the nature of this situation as neither can't I read the file to infer the error nor debug.

Any tips on how should I debug this? As I said the facets data are correct. What intrigues me is that some software can't open it and others can, but process just approximately half of the facets.

[jamie]

Maybe you have bad data ?

All values must be +, no - values and no zero values..

 Maybe a check should be made to ensure you aren't getting any - values or 0 values, this could be the error ?

[ChrisR]

If other tools are only showing half the facets, it suggests you are incorrectly specifying the normals or the winding order. Remember to use a counter-clockwise winding order for your triangles. The winding order discriminates the front from back face of a mesh.
  https://www.khronos.org/opengl/wiki/Face_Culling
I wonder if half your faces are showing as half are being culled do to incorrect winding order.

1. Does Surfice load these files. Can you share an example of a faulty file:
 https://github.com/neurolabusc/surf-ice/releases
If some triangles look matte relative to the others, you could check if choosing "MatCap2Side" makes the object look better. This would not fix your data, but it would diagnose the problem.

2. You could use a tool like Blender to attempt to correct your winding errors:
 https://www.shapeways.com/forum/t/how-to-fix-triangles-with-mixed-winding-order.15223/

[m4hunk]

Maybe you have bad data ?

All values must be +, no - values and no zero values..

 Maybe a check should be made to ensure you aren't getting any - values or 0 values, this could be the error ?

From my research, positive numbers were required in the past. Now it is common to find negative coordinates in STL files. I want to emphasize that I've written a code to create a ASCII stl file that works flawlessly with the same data that I am trying to write the Binary version. However what you stated shifted my attention, because as I said when I opened the file with netfabb approximately half of the facets does not show. Maybe they are not showing because they are the negative coordinates of the object?

 

If other tools are only showing half the facets, it suggests you are incorrectly specifying the normals or the winding order. Remember to use a counter-clockwise winding order for your triangles. The winding order discriminates the front from back face of a mesh.
  https://www.khronos.org/opengl/wiki/Face_Culling
I wonder if half your faces are showing as half are being culled do to incorrect winding order.

1. Does Surfice load these files. Can you share an example of a faulty file:
 https://github.com/neurolabusc/surf-ice/releases
If some triangles look matte relative to the others, you could check if choosing "MatCap2Side" makes the object look better. This would not fix your data, but it would diagnose the problem.

2. You could use a tool like Blender to attempt to correct your winding errors:
 https://www.shapeways.com/forum/t/how-to-fix-triangles-with-mixed-winding-order.15223/

I was careful to correctly specify the normals and the winding order. Specifically, I made sure that all the facets were winded in counter-clockwise way and the I had the same care with the vector products.

Regarding Surfice it said that the file is too small to open. The file is too big to share in the forum, so I used a website to share files. https://ufile.io/wtp7ncjv

[ChrisR]

Yes, your file is certainly too small for the number of triangles. The STL format has an 84 byte header, and the minimum number of bytes per triangle is 50 bytes (3 32-bit floats for facet normal, 9 32-bit floats for the vertex positions, 1 16-bit integer for "attribute byte count"). Your file clearly claims to store 103740 triangles, so we expect
  84 + (50 * 103740) = 5187084 bytes of data = 5.19 mb
but the sample file you shared is only 1910284 bytes = 1.91mb.

I have to say, I am very unfamiliar with how your program uses FileWrite(). This does look pretty archaic to me. I would suggest you try using blockwrite() as shown below - I have often used this for large binary files. Alternatively, my prior code looks uses TMemoryStream, which also works. As a sanity check, you should make sure your program generates a file of the expected size.

Here is how I would write a binary file:


  f:file;
begin
  assign(f,'BINARY_STL.stl') ;
  rewrite(f,1);
  blockwrite(f,a,sizeof(a));
  blockwrite(f,b,sizeof(b));
  ...
  close(f);
end;

[m4hunk]

Yes, your file is certainly too small for the number of triangles. The STL format has an 84 byte header, and the minimum number of bytes per triangle is 50 bytes (3 32-bit floats for facet normal, 9 32-bit floats for the vertex positions, 1 16-bit integer for "attribute byte count"). Your file clearly claims to store 103740 triangles, so we expect
  84 + (50 * 103740) = 5187084 bytes of data = 5.19 mb
but the sample file you shared is only 1910284 bytes = 1.91mb.

I have to say, I am very unfamiliar with how your program uses FileWrite(). This does look pretty archaic to me. I would suggest you try using blockwrite() as shown below - I have often used this for large binary files. Alternatively, my prior code looks uses TMemoryStream, which also works. As a sanity check, you should make sure your program generates a file of the expected size.

Here is how I would write a binary file:


  f:file;
begin
  assign(f,'BINARY_STL.stl') ;
  rewrite(f,1);
  blockwrite(f,a,sizeof(a));
  blockwrite(f,b,sizeof(b));
  ...
  close(f);
end;

I tried using blockwrite before trying this approach but I was unsuccessful. That's why I tried to use Filewrite. Thanks ChrisR, I will try to use your suggestion again! Regarding the file size, yes I always knew the problem was with the writing code, but I couldn't figure it out what was the problem. I'll let you know when I finish.

[ChrisR]

You need to change

Code: Pascal  [Select][+][-]

1.   attribute_byteCount, i : UInt16;

to read

Code: Pascal  [Select][+][-]

1.   attribute_byteCount : UInt16;
2.   i: UInt32;

Your current code will fail for meshes with more than 65,535 triangles, as a UInt16 is in the range 0..65,535.

[jamie]

I guess that throws the standards out the door!

[ChrisR]

@jamie. Nothing wrong with the standard. It specifies "Number of facets in file" is saved as a UINT32, and that is correctly written as the value intBuffer. The issue with the original example is that the value "i" that was used to loop through each facet was defined as Uint16, which means the code will break if asked to save more than 65535 facets. Changing "i" to be UInt32 allows the loop to succeed to the limit allowed by the format.

Code: Pascal  [Select][+][-]

1. var
2.   attribute_byteCount, i : UInt16;
3.   intBuffer            : UInt32;
4. ...
5.  FileWrite(F,intBuffer,sizeof(intBuffer));
6.  
7.   for i:=0 to intBuffer do
8.  

[m4hunk]

You need to change

Code: Pascal  [Select][+][-]

1.   attribute_byteCount, i : UInt16;

to read

Code: Pascal  [Select][+][-]

1.   attribute_byteCount : UInt16;
2.   i: UInt32;

Your current code will fail for meshes with more than 65,535 triangles, as a UInt16 is in the range 0..65,535.

Thank you ChrisR! It worked! I didn't think the problem would be such simple and conceptual. I was so focused in finding a more complex bug that I'd forgotten to pay attention to the basic concepts, I will pay more attention in the future.

@Edit: I will leave the topic open for some time in case anyone wants to comment on the topic.