Visitar URL original
c_uint32 bitfields break structures · Issue #68478 · python/cpython · GitHub
Skip to content

c_uint32 bitfields break structures #68478

Closed
Closed
c_uint32 bitfields break structures#68478
Labels
topic-ctypestype-bugAn unexpected behavior, bug, or error

Description

@RonyBatista
RonyBatista
mannequin
opened on May 26, 2015
BPO 24290
Nosy @ronaldoussoren, @ned-deily
Files
  • ctypesBug.py
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2015-05-26.19:02:48.705>
labels = ['ctypes', 'type-bug']
title = 'c_uint32 bitfields break structures'
updated_at = <Date 2015-05-27.22:57:25.193>
user = 'https://bugs.python.org/RonyBatista'

    bugs.python.org fields:

    activity = <Date 2015-05-27.22:57:25.193>
actor = 'Rony Batista'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['ctypes']
creation = <Date 2015-05-26.19:02:48.705>
creator = 'Rony Batista'
dependencies = []
files = ['39508']
hgrepos = []
issue_num = 24290
keywords = []
message_count = 5.0
messages = ['244125', '244126', '244127', '244132', '244223']
nosy_count = 3.0
nosy_names = ['ronaldoussoren', 'ned.deily', 'Rony Batista']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue24290'
versions = ['Python 2.7']

    Activity

    1. RonyBatista commented on May 26, 2015

      @RonyBatista
      RonyBatistamannequin
      on May 26, 2015
      MannequinAuthor

      ctypes Structures with c_uint32 bitfields have strange behaviour on OS X.

      In the attached code when the field "number" is set, it changes the whole 32 bits, even thou its meant to be 23 bits.

      The behavior is unexpected in:
      OS X: Python 2.7.9

      The behavior is as expected in the following environments:
      Windows: python 2.7.9
      Ubuntu: python 2.7.6

    2. RonyBatista
      added
      topic-ctypes
      type-bugAn unexpected behavior, bug, or error
      on May 26, 2015

    3. ronaldoussoren commented on May 26, 2015

      @ronaldoussoren
      ronaldoussoren
      on May 26, 2015
      Contributor

      You forgot to actually attach the code.

    4. RonyBatista commented on May 26, 2015

      @RonyBatista
      RonyBatistamannequin
      on May 26, 2015
      MannequinAuthor

      Silly me, Heres the code.

    5. ned-deily commented on May 26, 2015

      @ned-deily
      ned-deily
      on May 26, 2015
      Member

      Without diving into the details of your test program, I get the same results on a 64-bit Debian Python 2.7.9 as with a 64-bit OS X 2.7.10:

      c_uint32 TESTS:
      Class Name exponent number Sign float binary repr
      IEEE754Float_u ('-0x7b', '0xbeb2aL', 0L, 69.95930480957031, ['0x42', '0x8b', '0xeb', '0x2a'])
      IEEE754Float_u ('-0x7b', '0xbeb2aL', 0L, 69.95930480957031, ['0x42', '0x8b', '0xeb', '0x2a'])
      AN575Float_uint ('0x0', '0xbeb2a', 0L, 1.094551427887186e-39, ['0x0', '0xb', '0xeb', '0x2a'])
      AN575Float_uint ('0x0', '0xbeb2a', 0L, 1.094551427887186e-39, ['0x0', '0xb', '0xeb', '0x2a'])
      AN575Float_uint ('-0x7b', '0xbeb2a', 0L, 1.094551427887186e-39, ['0x85', '0xb', '0xeb', '0x2a'])
      IEEE754Float_u ('-0x7b', '0xbeb2aL', 1L, -69.95930480957031, ['0xc2', '0x8b', '0xeb', '0x2a'])
      AN575Float_uint ('-0x7b', '0xbeb2a', 1L, 1.094551427887186e-39, ['0x85', '0x8b', '0xeb', '0x2a'])

      But using the same OS X 2.7.10 in 32-bit mode, I get:

      c_uint32 TESTS:
      Class Name exponent number Sign float binary repr
      IEEE754Float_u ('-0x7b', '0xbeb2aL', 0L, 69.95930480957031, ['0x42', '0x8b', '0xeb', '0x2a'])
      IEEE754Float_u ('-0x7b', '0xbeb2aL', 0L, 69.95930480957031, ['0x42', '0x8b', '0xeb', '0x2a'])
      AN575Float_uint ('-0x7b', '0xbeb2a', 0L, 69.95930480957031, ['0x85', '0xb', '0xeb', '0x2a'])
      AN575Float_uint ('-0x7b', '0xbeb2a', 0L, 69.95930480957031, ['0x85', '0xb', '0xeb', '0x2a'])
      AN575Float_uint ('-0x7b', '0xbeb2a', 0L, 69.95930480957031, ['0x85', '0xb', '0xeb', '0x2a'])
      IEEE754Float_u ('-0x7b', '0xbeb2aL', 1L, -69.95930480957031, ['0xc2', '0x8b', '0xeb', '0x2a'])
      AN575Float_uint ('-0x7b', '0xbeb2a', 1L, -69.95930480957031, ['0x85', '0x8b', '0xeb', '0x2a'])

    6. RonyBatista commented on May 27, 2015

      @RonyBatista
      RonyBatistamannequin
      on May 27, 2015
      MannequinAuthor

      Well, looks like the issue is with 64 bit mode then.

      For the first 5 cases the right answer is 69.95930480957031, and for the last 2 its -69.95930480957031. The results for the 32 bit mode are all correct.

    7. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    8. arhadthedev
      added
      OS-mac
      on Apr 20, 2023
    9. ronaldoussoren
      removed
      OS-mac
      on Jun 24, 2023
    10. serhiy-storchaka
      added this to Ctypes issueson Jan 1, 2024

    11. encukou commented on May 13, 2025

      @encukou
      encukou
      on May 13, 2025
      Member

      Should be fixed in #97702.

      Code updated for Python 3:

      __author__ = 'rony'

from ctypes import BigEndianStructure
from ctypes import sizeof, memmove, addressof
from ctypes import c_int32, c_uint32
from struct import pack, unpack


class BigEndianByteStructure(BigEndianStructure):
    # https://wiki.python.org/moin/ctypes
    @classmethod
    def from_bytes(cls, ibytes):
        return_val = cls()
        if len(ibytes) != sizeof(return_val):
            raise ValueError(
                '{0} Cannot unpack {2} bytes, it should be {1} bytes'.format(cls.__name__, sizeof(return_val),
                                                                             len(ibytes)))

        memmove(addressof(return_val), ibytes, len(ibytes))
        return return_val

    @classmethod
    def from_structure(cls, original):
        return_val = cls()
        for field in return_val._fields_:
            try:
                setattr(return_val, field[0], getattr(original, field[0]))
            except:
                print( 'Attribute {0} not found in original'.format(field[0]))

        return return_val


# POSITIVE
# IEEE754 HEX    0x428beb2a
# AN575 HEX      0x850beb2a

# NEGATIVE
# IEEE754 HEX    0xc28beb2a
# AN575 HEX      0x858beb2a

class IEEE754Float(BigEndianByteStructure):
    _pack_ = 1
    _fields_ = [('sign', c_int32, 1),
                ('exponent', c_int32, 8),
                ('number', c_int32, 23)]

    @classmethod
    def from_float(cls, f):
        return cls.from_bytes(pack('>f', f))

    def __float__(self):
        return unpack('>f', memoryview(self))[0]

    def debug(self):
        f = self
        return hex(f.exponent), hex(f.number), f.sign, float(f), [hex(b) for b in bytearray(f)]


class AN575Float(BigEndianByteStructure):
    _pack_ = 1
    _fields_ = [('exponent', c_int32, 8),
                ('sign', c_int32, 1),
                ('number', c_int32, 23)]

    @classmethod
    def from_float(cls, f):
        return cls.from_structure(IEEE754Float.from_float(f))

    def __float__(self):
        a = IEEE754Float.from_structure(self)
        return float(a)

    def debug(self):
        f = self
        return hex(f.exponent), hex(f.number), f.sign, float(f), [hex(b) for b in bytearray(f)]


class IEEE754Float_u(BigEndianByteStructure):
    _pack_ = 1
    _fields_ = [('sign', c_uint32, 1),
                ('exponent', c_int32, 8),
                ('number', c_uint32, 23)]

    @classmethod
    def from_float(cls, f):
        return cls.from_bytes(pack('>f', f))

    def __float__(self):
        return unpack('>f', memoryview(self))[0]

    def debug(self):
        f = self
        return hex(f.exponent), hex(f.number), f.sign, float(f), [hex(b) for b in bytearray(f)]


class AN575Float_uint(BigEndianByteStructure):
    _pack_ = 1
    _fields_ = [('exponent', c_int32, 8),
                ('sign', c_uint32, 1),
                ('number', c_int32, 23)]

    @classmethod
    def from_float(cls, f):
        return cls.from_structure(IEEE754Float_u.from_float(f))

    def __float__(self):
        a = IEEE754Float_u.from_structure(self)
        return float(a)

    def debug(self):
        f = self
        return hex(f.exponent), hex(f.number), f.sign, float(f), [hex(b) for b in bytearray(f)]


print ('\n', 'c_int32 TESTS:')
print ('Class Name\t\t', 'exponent', '\tnumber', '\tSign', '\tfloat', '\t\t\t\t', 'binary repr')
f = IEEE754Float.from_float(69.9593067)
print (type(f).__name__, f.debug())

f = IEEE754Float.from_bytes(b'\x42\x8b\xeb\x2a')
print (type(f).__name__, f.debug())

f = AN575Float.from_float(69.9593067)
print (type(f).__name__, f.debug())

f = AN575Float.from_structure(IEEE754Float.from_float(69.9593067))
print (type(f).__name__, f.debug())

f = AN575Float.from_bytes(b'\x85\x0b\xeb\x2a')
print (type(f).__name__, f.debug())

f = IEEE754Float.from_bytes(b'\xc2\x8b\xeb\x2a')
print (type(f).__name__, f.debug())

f = AN575Float.from_bytes(b'\x85\x8b\xeb\x2a')
print (type(f).__name__, f.debug())

print ('\n', 'c_uint32 TESTS:')
print ('Class Name\t\t', 'exponent', '\tnumber', '\tSign', '\tfloat', '\t\t\t\t', 'binary repr')
f = IEEE754Float_u.from_float(69.9593067)
print (type(f).__name__, f.debug())

f = IEEE754Float_u.from_bytes(b'\x42\x8b\xeb\x2a')
print (type(f).__name__, f.debug())

f = AN575Float_uint.from_float(69.9593067)
print (type(f).__name__, f.debug())

f = AN575Float_uint.from_structure(IEEE754Float.from_float(69.9593067))
print (type(f).__name__, f.debug())

f = AN575Float_uint.from_bytes(b'\x85\x0b\xeb\x2a')
print (type(f).__name__, f.debug())

f = IEEE754Float_u.from_bytes(b'\xc2\x8b\xeb\x2a')
print (type(f).__name__, f.debug())

f = AN575Float_uint.from_bytes(b'\x85\x8b\xeb\x2a')
print (type(f).__name__, f.debug())

      Result (same on 64- and 32-bit OS):

      
 c_int32 TESTS:
Class Name		 exponent 	number 	Sign 	float 				 binary repr
IEEE754Float ('-0x7b', '0xbeb2a', 0, 69.95930480957031, ['0x42', '0x8b', '0xeb', '0x2a'])
IEEE754Float ('-0x7b', '0xbeb2a', 0, 69.95930480957031, ['0x42', '0x8b', '0xeb', '0x2a'])
AN575Float ('-0x7b', '0xbeb2a', 0, 69.95930480957031, ['0x85', '0xb', '0xeb', '0x2a'])
AN575Float ('-0x7b', '0xbeb2a', 0, 69.95930480957031, ['0x85', '0xb', '0xeb', '0x2a'])
AN575Float ('-0x7b', '0xbeb2a', 0, 69.95930480957031, ['0x85', '0xb', '0xeb', '0x2a'])
IEEE754Float ('-0x7b', '0xbeb2a', -1, -69.95930480957031, ['0xc2', '0x8b', '0xeb', '0x2a'])
AN575Float ('-0x7b', '0xbeb2a', -1, -69.95930480957031, ['0x85', '0x8b', '0xeb', '0x2a'])

 c_uint32 TESTS:
Class Name		 exponent 	number 	Sign 	float 				 binary repr
IEEE754Float_u ('-0x7b', '0xbeb2a', 0, 69.95930480957031, ['0x42', '0x8b', '0xeb', '0x2a'])
IEEE754Float_u ('-0x7b', '0xbeb2a', 0, 69.95930480957031, ['0x42', '0x8b', '0xeb', '0x2a'])
AN575Float_uint ('-0x7b', '0xbeb2a', 0, 69.95930480957031, ['0x85', '0xb', '0xeb', '0x2a'])
AN575Float_uint ('-0x7b', '0xbeb2a', 0, 69.95930480957031, ['0x85', '0xb', '0xeb', '0x2a'])
AN575Float_uint ('-0x7b', '0xbeb2a', 0, 69.95930480957031, ['0x85', '0xb', '0xeb', '0x2a'])
IEEE754Float_u ('-0x7b', '0xbeb2a', 1, -69.95930480957031, ['0xc2', '0x8b', '0xeb', '0x2a'])
AN575Float_uint ('-0x7b', '0xbeb2a', 1, -69.95930480957031, ['0x85', '0x8b', '0xeb', '0x2a'])
    12. encukou
      closed this as completedon May 13, 2025
    13. github-project-automation
      moved this to Done in Ctypes issueson May 13, 2025
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      topic-ctypestype-bugAn unexpected behavior, bug, or error

      Projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions