Skip to content

Powerinfer's server-side babble has occurred while the command-side looks good #269

Description

@Chosen-David

Prerequisites

Before submitting your question, please ensure the following:

  • I am running the latest version of PowerInfer. Development is rapid, and as of now, there are no tagged versions.
  • I have carefully read and followed the instructions in the README.md.
  • I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).

Question Details

Image

Powerinfer's server-side babble has occurred

My configuration is as follows

Image

The commands I run are as follows

./build/bin/server -m /PowerInfer/path/ReluLLaMA-7B.powerinfer.gguf

Can someone explain to me why, or is that why the server side is not written right

Additional Context

Image

It is worth noting that my reasoning works fine when running locally, but there is some gibberish on the server.

The commands I run on the command line are as follows

./build/bin/main -m /home/yswang/AI_LLM/PowerInfer/models/GGUF/ReluLLaMA-7B.powerinfer.gguf -n 128 -t 8 -p "Once upon a time"

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    questionFurther information is requested

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions