System.OverflowException is thrown when LZ4MessagePackSerializer.Deserialize

Bug description

Our team use LZ4MessagePackSerializer.Serialize to serialize objects to file.
And deserialize objects by using LZ4MessagePackSerializer.Deserialize.

But one day, when deserializing, an System.OverflowException is thrown.
Why doesn't it throw the exception when serializing and saving?

Repro steps

  using (Stream stream = File.OpenRead(fileName))
  {
      stream.Seek(0, SeekOrigin.Begin);
      var localFileData = LZ4MessagePackSerializer.Deserialize<T>(stream); //object
  }

Expected behavior

no exceptions are thrown.

Actual behavior

one System.OverflowException is thrown.

Version used: 1.9.0.0
Runtime: .NET Framework 4.5.2

Additional context

Exception Stacktrace:

System.OverflowException: 算术运算导致溢出。
在 MessagePack.LZ4.LZ4Codec.CheckArguments(Byte[] input, Int32 inputOffset, Int32 inputLength, Byte[] output, Int32 outputOffset, Int32 outputLength)
在 MessagePack.LZ4.LZ4Codec.Decode64Unsafe(Byte[] input, Int32 inputOffset, Int32 inputLength, Byte[] output, Int32 outputOffset, Int32 outputLength)
在 MessagePack.LZ4MessagePackSerializer.DeserializeCore[T](ArraySegment`1 bytes, IFormatterResolver resolver)
在 MessagePack.LZ4MessagePackSerializer.Deserialize[T](Stream stream, IFormatterResolver resolver, Boolean readStrict)

My best guess is your file was corrupted between being written and read. If you can provide a full repro that includes serializing and the failed deserialization we can investigate further.

Hi. Our team encountered similar issue too. Using MessagePackSerializer.Typeless.Deserialize with MessagePackCompression.Lz4BlockArray throws in MessagePack.MessagePackSerializer.DeserializeFromSequenceAndRewindStreamIfPossible when underlying FileStream size is above int.MaxValue. Shall I provide you a repro project or is it enough to look into it? I could also open a new issue if it doesn't seem related enough.

Used NuGet package version: 2.5.124.

Stack trace:

Unhandled exception. System.OverflowException: Arithmetic operation resulted in an overflow.
   at MessagePack.MessagePackSerializer.DeserializeFromSequenceAndRewindStreamIfPossible[T](Stream streamToRewind, MessagePackSerializerOptions options, ReadOnlySequence`1 sequence, CancellationToken cancellationToken)
   at MessagePack.MessagePackSerializer.Deserialize[T](Stream stream, MessagePackSerializerOptions options, CancellationToken cancellationToken)
   at MessagePack.MessagePackSerializer.Typeless.Deserialize(Stream stream, MessagePackSerializerOptions options, CancellationToken cancellationToken)

@gosferano

@gosferano You've come across a long to int cast that doesn't even need to be there. I've removed it, but as I don't have any 4GB+ streams to deserialize handy to test with, can you help validate it by trying out the build in the #1685 PR?

@AArnott

@AArnott I tried #1685 on repro project I had and the problem is fixed.

Shipped in v2.5.129 to nuget.org.

Navigation Menu

Repository navigation

System.OverflowException is thrown when LZ4MessagePackSerializer.Deserialize #1586

Bug description

Repro steps

Expected behavior

Actual behavior

Additional context

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

System.OverflowException is thrown when LZ4MessagePackSerializer.Deserialize #1586

Description

Bug description

Repro steps

Expected behavior

Actual behavior

Additional context

Activity

AArnott commented on Apr 6, 2023

gosferano commented on Sep 26, 2023

AArnott commented on Sep 27, 2023

gosferano commented on Sep 28, 2023

AArnott commented on Sep 28, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions