Visitar URL original
context memory pool · Issue #3 · bigcode-project/starcoder.cpp · GitHub
Skip to content

context memory pool #3

Closed
#6
Closed
context memory pool#3
#6

Description

@Leterax
Leterax
opened on May 16, 2023

When running the StarCoder model quantized using "q5_1" with a medium-sized context (3500 tokens) I run into this error:

ggml_new_tensor_impl: not enough space in the context's memory pool (needed 412241472, available 411791504)
Segmentation fault (core dumped)

How can I increase the context memory pool size?

Activity

  1. nmntz commented on May 20, 2023

    @nmntz
    nmntz
    on May 20, 2023
    Collaborator

    Yes, I'm able to reproduce this issue! We're investigating a fix here: ggml-org/ggml#175

  2. nmntz
    closed this as completedin #6on May 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions