Visitar URL original
[sqlite3] Do not truncate results of user functions and aggregates on the first NUL · Issue #88985 · python/cpython · GitHub
Skip to content

[sqlite3] Do not truncate results of user functions and aggregates on the first NUL #88985

Closed
Closed
[sqlite3] Do not truncate results of user functions and aggregates on the first NUL#88985
Assignees
erlend-aasland
Labels
3.10 (EOL)end of life3.11only security fixes3.9 (EOL)end of lifeextension-modulesC modules in the Modules dirtopic-sqlite3type-bugAn unexpected behavior, bug, or error

Description

@erlend-aasland
erlend-aasland
opened on Aug 4, 2021
BPO 44822
Nosy @serhiy-storchaka, @miss-islington, @erlend-aasland
PRs
  • bpo-44822: Don't truncate strs with embedded NULL chars returned by sqlite3 UDF callbacks #27588
  • [3.10] bpo-44822: Don't truncate strs with embedded NULL chars returned by sqlite3 UDF callbacks (GH-27588) #27611
  • [3.9] bpo-44822: Don't truncate strs with embedded NULL chars returned by sqlite3 UDF callbacks (GH-27588) #27639
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/erlend-aasland'
closed_at = <Date 2021-08-06.21:08:39.691>
created_at = <Date 2021-08-04.08:29:38.206>
labels = ['extension-modules', 'type-bug', '3.9', '3.10', '3.11']
title = '[sqlite3] Do not truncate results of user functions and aggregates on the first NUL'
updated_at = <Date 2021-08-06.21:08:39.690>
user = 'https://github.com/erlend-aasland'

    bugs.python.org fields:

    activity = <Date 2021-08-06.21:08:39.690>
actor = 'erlendaasland'
assignee = 'erlendaasland'
closed = True
closed_date = <Date 2021-08-06.21:08:39.691>
closer = 'erlendaasland'
components = ['Extension Modules']
creation = <Date 2021-08-04.08:29:38.206>
creator = 'erlendaasland'
dependencies = []
files = []
hgrepos = []
issue_num = 44822
keywords = ['patch']
message_count = 8.0
messages = ['398865', '398869', '398873', '398881', '398971', '399115', '399146', '399149']
nosy_count = 3.0
nosy_names = ['serhiy.storchaka', 'miss-islington', 'erlendaasland']
pr_nums = ['27588', '27611', '27639']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue44822'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

    Activity

    1. erlend-aasland commented on Aug 4, 2021

      @erlend-aasland
      erlend-aasland
      on Aug 4, 2021
      ContributorAuthor

      The third argument to sqlite3_result_text() is the length of the string passed as the second argument. Currently, we pass -1, so SQLite has to invoke strlen() to compute the length of the passed string. Suggesting to use PyUnicode_AsUTF8AndSize() iso. PyUnicode_AsUTF8() and pass the string size to avoid the superfluous strlen().

      See also:

    2. erlend-aasland
      added
      type-featureA feature request or enhancement
      3.11only security fixes
      on Aug 4, 2021
    3. erlend-aasland
      self-assigned this
      on Aug 4, 2021
    4. erlend-aasland
      added
      type-featureA feature request or enhancement
      extension-modulesC modules in the Modules dir
      3.11only security fixes
      on Aug 4, 2021
    5. erlend-aasland
      self-assigned this
      on Aug 4, 2021
    6. erlend-aasland
      added
      extension-modulesC modules in the Modules dir
      on Aug 4, 2021

    7. serhiy-storchaka commented on Aug 4, 2021

      @serhiy-storchaka
      serhiy-storchaka
      on Aug 4, 2021
      Member

      The difference between specifying negative and non-negative third argument of sqlite3_result_text() is that in the latter case the result can contain embedded NUL characters.

      Could you please add a test for string containing embedded NUL?

      Letting SQLite compute string length would work incorrect in case of embedded NULs, so I think that we should raise explicit error if sz > INT_MAX.

    8. erlend-aasland commented on Aug 4, 2021

      @erlend-aasland
      erlend-aasland
      on Aug 4, 2021
      ContributorAuthor

      Could you please add a test for string containing embedded NUL?

      Of course, thanks for the heads up.

      I think that we should raise explicit error if sz > INT_MAX.

      Yes, I thought about this. I do agree that raising OverflowError would be better. Thanks.

    9. erlend-aasland commented on Aug 4, 2021

      @erlend-aasland
      erlend-aasland
      on Aug 4, 2021
      ContributorAuthor

      Serhiy, I've updated the PR, if you want to take a look. Thanks for your feedback.

    10. serhiy-storchaka
      added
      3.9 (EOL)end of life
      3.10 (EOL)end of life
      on Aug 4, 2021
    11. serhiy-storchaka
      changed the title [-][sqlite3] Micro-optimisation: pass string size to sqlite3_result_text()[/-] [+][sqlite3] Do not truncate results of user functions and aggregates on the first NUL[/+] on Aug 4, 2021
    12. serhiy-storchaka
      added
      3.9 (EOL)end of life
      3.10 (EOL)end of life
      type-bugAn unexpected behavior, bug, or error
      and removed
      type-featureA feature request or enhancement
      on Aug 4, 2021
    13. serhiy-storchaka
      changed the title [-][sqlite3] Micro-optimisation: pass string size to sqlite3_result_text()[/-] [+][sqlite3] Do not truncate results of user functions and aggregates on the first NUL[/+] on Aug 4, 2021
    14. serhiy-storchaka
      added
      type-bugAn unexpected behavior, bug, or error
      and removed
      type-featureA feature request or enhancement
      on Aug 4, 2021

    15. serhiy-storchaka commented on Aug 5, 2021

      @serhiy-storchaka
      serhiy-storchaka
      on Aug 5, 2021
      Member

      New changeset 8f010dc by Erlend Egeberg Aasland in branch 'main':
      bpo-44822: Don't truncate strs with embedded NULL chars returned by sqlite3 UDF callbacks (GH-27588)
      8f010dc

    16. miss-islington commented on Aug 6, 2021

      @miss-islington
      miss-islington
      on Aug 6, 2021
      Contributor

      New changeset 2b1e713 by Miss Islington (bot) in branch '3.10':
      bpo-44822: Don't truncate strs with embedded NULL chars returned by sqlite3 UDF callbacks (GH-27588)
      2b1e713

    17. serhiy-storchaka commented on Aug 6, 2021

      @serhiy-storchaka
      serhiy-storchaka
      on Aug 6, 2021
      Member

      New changeset c352412 by Erlend Egeberg Aasland in branch '3.9':
      [3.9] bpo-44822: Don't truncate strs with embedded NULL chars returned by sqlite3 UDF callbacks (GH-27588). (GH-27639)
      c352412

    18. erlend-aasland commented on Aug 6, 2021

      @erlend-aasland
      erlend-aasland
      on Aug 6, 2021
      ContributorAuthor

      Thanks, Serhiy!

    19. erlend-aasland
      closed this as completedon Aug 6, 2021
    20. erlend-aasland
      closed this as completedon Aug 6, 2021
    21. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    22. erlend-aasland
      added
      topic-sqlite3
      on May 16, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    Labels

    3.10 (EOL)end of life3.11only security fixes3.9 (EOL)end of lifeextension-modulesC modules in the Modules dirtopic-sqlite3type-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions