Extracting all lines (in the generic sense) from DXF

[russell-sealand]

Hi,

I'm not the best Python programmer, but I have a problem I'm hoping to get some help with. I have a 2D plan view of pipelines etc (i.e. a 2D map with a series of polylines) in DXF, however when I've tried to export using ezdxf and my own code I only get some of the polylines, not the complete set despite them all being identified as polylines in AutoCAD. I was wondering if they were different entity types so I've been trying to export any coordinates I can get hold of.

Original AutoCAD DXF:

Export using code below:

Can anyone please give me any guidance on what I need to try?

Here is a copy of my hacky loop that tries to get any kind of coordinate it can and dump it to my Excel file:

for entity in msp:
print('Entity #{}'.format(entity.dxf.handle))

# If it's a LINE
if entity.dxftype() == 'LINE':
    worksheet.write(row, col, entity.dxf.start[0])
    worksheet.write(row, col+1, entity.dxf.start[0])
    print("** WRITING VERTICES")
    row += 1

    worksheet.write(row, col, entity.dxf.end[0])
    worksheet.write(row, col, entity.dxf.end[1])
    print("** WRITING VERTICES")
    row += 1

try:
    for i, location in enumerate(entity.vertices()):
        worksheet.write(row, col, location[0])
        worksheet.write(row, col+1, location[1])
        worksheet.write(row, col+2, entity.dxf.handle)
        print("** WRITING VERTICES")
        row += 1
except:
    print("Entity had no Vertices")

try:
    for i, location in enumerate(entity.points()):
        worksheet.write(row, col, location.x)
        worksheet.write(row, col+1, location.y)
        worksheet.write(row, col+2, entity.dxf.handle)
        print("** WRITING POINTS")
        row += 1
except:
    print("Entity had no Polyline Points, moving to next entity")

Thanks,

Russell.

[mozman]

Remark to your polyline processing:
Do not use method vertices() without type testing:
Polyline.vertices() returns an iterable of VERTEX entities, LWPolyline.vertices() returns an iterable of (x, y) tuples, luckily your DXF file does not contain POLYLINE entities. I know the POLYLINE and LWPOLYLINE interface inconsistency is a real mess and will be tackled in the future.

The possible problems:

1. Some lines are SPLINE entities
2. Lines are located in block references (INSERT entities)

import ezdxf

def process_lines(entities):
    for e in entities:
        print(str(e))
        dxftype = e.dxftype()
        if dxftype == "LINE":
            print(e.dxf.start)
            print(e.dxf.end)

        elif dxftype == "POLYLINE":
            if e.is_2d_polyline or e.is_3D_polyline:
                # other types: is_polygon_mesh and is_poly_face_mesh
                # 2D polylines can have an extrusion != (0, 0, 1) and bulges 
                # see LWPOLYLINE processing.
                for point in e.points():
                    print(point)

        elif dxftype == "LWPOLYLINE":
            # vertices_in_wcs() returns also correct WCS locations for LWPOLYLINE
            # entities placed in 3D space, OCS with extrusion != (0, 0, 1).
            # An extrusion (0, 0, -1) can also happen by mirroring block references!
            # LWPolylines can also contain bulges (arcs), in this case a bulge 
            # is processed as straight line.
            for point in e.vertices_in_wcs():
                print(point)

        elif dxftype == "SPLINE":
            # flatten curve with max distance of 0.01 drawing units between
            # spline and approximation polyline.
            for point in e.flattening(0.01):
                print(point)


def process(entities):
    process_lines(entities)

    # recursive processing of block references:
    for e in entities:
        if e.dxftype() == 'INSERT':
            # create virtual entities based on the block definition
            # and store the content in a list, virtual_entities() returns 
            # a generator. The content will be transformed into the parent 
            # coordinate system:
            process(list(e.virtual_entities()))

doc = ezdxf.readfile("your.dxf")

# Start processing at the top level: model space
process(doc.modelspace())

NEW in V0.16 (not released yet, beta version on PyPI): disassemble DXF entities into primitives

import ezdxf
from ezdxf import disassemble

doc = ezdxf.readfile("your.dxf")
msp = doc.modelspace()

# recursive_decompose() dissolves nested layout structures into a flat stream of
# DXF entities:
lines = (  # generator
    e for e in disassemble.recursive_decompose(msp)
    if e.dxftype() in {'LINE', 'POLYLINE', 'LWPOLYLINE', 'SPLINE'}
)
for e in lines:
    print(str(e))
    # create a primitive with a common interface, flattens bulges
    # into polylines and handles OCS for 2D entities:
    p = disassemble.make_primitive(e, max_flattening_distance=0.01)
    for vertex in p.vertices():
        print(vertex)

[russell-sealand]

Thanks for your prompt response, I went for option 2 since this is much clearer to me.

The code increases my saved set of vertices from 19,000 to 75,000. If I slice it and take every 200th set of coordinates and export to my other software (which can't handle the whole dataset) unfortunately the picture looks almost exactly the same as before.

I get an exception for "for vertex in p.vertices():" periodically, which I bypass with a try/except, however maybe this is key to the missing lines? Exception as follows:

Exception has occurred: ValueError
math domain error
  File "...1 DXF to XLSX.py", line 49, in <module>
    for vertex in p.vertices():

--

Can you explain the max_flattening_distance parameter a little more? Is it perhaps generating too many component lines? Note I am working in the km range, so my coordinates are in the order of: x= 700555.6....., y = 5704368.5..... is a flattening distance of 0.01 much too small?

[mozman]

Max flattening distance of 0.1 for a circle with radius 1 (all in drawing units):

The smaller this value is, the more approximation lines are required.

Flattening of the same circle and a max flattening distance of 0.02:

There should be no exceptions, can you provide your DXF file for bug fixing?

[russell-sealand]

It's 145mb, please download it from the following link DXF file

[mozman]

Thank you!

[mozman]

FYI: DXF can be very good compressed - 7zip compresses your file down to 12MB (.7z)

[mozman]

I fixed the error caused by SPLINE entities, but only 517 SPLINE entities were skipped.

These are all existing entity types in your DXF file: {'INSERT', 'TEXT', 'LINE', 'CIRCLE', 'ELLIPSE', 'LWPOLYLINE', 'HATCH', 'ARC', 'SPLINE', 'POLYLINE'}

You can also include CIRCLE, ARC and ELLIPSE into the entity filter for flattening, maybe these are the missing objects.

[mozman]

These are really all entity types in your DXF file:

Counter({'LINE': 267434,
         'SPLINE': 27011,
         'ELLIPSE': 15650,
         'ARC': 14422,
         'CIRCLE': 13661,
         'LWPOLYLINE': 4337,
         'VERTEX': 3194,
         'ACAD_PROXY_OBJECT': 2745,
         'DICTIONARY': 706,
         'SEQEND': 357,
         'INSERT': 353,
         'XRECORD': 305,
         'HATCH': 231,
         'BLOCK_RECORD': 223,
         'BLOCK': 223,
         'ENDBLK': 223,
         'SORTENTSTABLE': 156,
         'MTEXT': 133,
         'LTYPE': 93,
         'SCALE': 83,
         'APPID': 77,
         'ACDB_BLOCKREPRESENTATION_DATA': 76,
         'LAYER': 58,
         'VISUALSTYLE': 26,
         'DICTIONARYVAR': 15,
         'MULTILEADER': 14,
         'ACAD_EVALUATION_GRAPH': 13,
         'ACDB_DYNAMICBLOCKPURGEPREVENTER_VERSION': 13,
         'BLOCKBASEPOINTPARAMETER': 13,
         'POINT': 12,
         'STYLE': 11,
         'VIEWPORT': 11,
         'TABLE': 9,
         'SOLID': 9,
         'SURFACE': 8,
         'TEXT': 7,
         'MATERIAL': 3,
         'POLYLINE': 2,
         'DIMENSION': 2,
         'ACDBDETAILVIEWSTYLE': 2,
         'LAYOUT': 2,
         'ACDBSECTIONVIEWSTYLE': 2,
         'TABLESTYLE': 2,
         'DATATABLE': 2,
         'CELLSTYLEMAP': 2,
         'VPORT': 1,
         'DIMSTYLE': 1,
         'RASTERVARIABLES': 1,
         'ACDBDICTIONARYWDFLT': 1,
         'WIPEOUTVARIABLES': 1,
         'GEODATA': 1,
         'MLEADERSTYLE': 1,
         'MLINESTYLE': 1,
         'ACDBPLACEHOLDER': 1})

This is an Autodesk Civil 3D file which contains many undocumented DXF entities and proprietary data embedded in 2745 ACAD_PROXY_OBJECTs. This all is beyond the DXF reference and the point where ezdxf can help you, sorry.

[russell-sealand]

Hi Manfred, thanks for the effort put into this, I appreciate it.

I tried adding CIRCLE, ARC and ELLIPSE but they are throwing exceptions, I would need your updated code to get past this I think? I also tried coding via your first method as well but hit the same 'math domain' issue when flattening splines. I think if I can get past this, then these entities should contain the missing data, it appears that way from looking at your entity breakdown list above.

Taking the long western line as an example, gives the following in AutoCAD LT 2021: