Instructions to use Sleem247/legal-ft-1-Q8_0-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use Sleem247/legal-ft-1-Q8_0-GGUF with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("Sleem247/legal-ft-1-Q8_0-GGUF") sentences = [ "Why is the use of AI systems particularly important for individuals applying for or receiving public assistance benefits?", "(48)", "Another area in which the use of AI systems deserves special consideration is the access to and enjoyment of certain essential private and public services and benefits necessary for people to fully participate in society or to improve one’s standard of living. In particular, natural persons applying for or receiving essential public assistance benefits and services from public authorities namely healthcare services, social security benefits, social services providing protection in cases such as maternity, illness, industrial accidents, dependency or old age and loss of employment and social and housing assistance, are typically dependent on those benefits and services and in a vulnerable position in relation to the responsible authorities.", "used for biometric verification, which includes authentication, the sole purpose of which is to confirm that a specific natural person is the person he or she claims to be and to confirm the identity of a natural person for the sole purpose of having access to a service, unlocking a device or having security access to premises. That exclusion is justified by the fact that such systems are likely to have a minor impact on fundamental rights of natural persons compared to the remote biometric identification systems which may be used for the processing of the biometric data of a large number of persons without their active involvement. In the case of ‘real-time’ systems, the capturing of the biometric data, the comparison and the" ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [4, 4] - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Sleem247/legal-ft-1-Q8_0-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Sleem247/legal-ft-1-Q8_0-GGUF:Q8_0 # Run inference directly in the terminal: llama cli -hf Sleem247/legal-ft-1-Q8_0-GGUF:Q8_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Sleem247/legal-ft-1-Q8_0-GGUF:Q8_0 # Run inference directly in the terminal: llama cli -hf Sleem247/legal-ft-1-Q8_0-GGUF:Q8_0
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf Sleem247/legal-ft-1-Q8_0-GGUF:Q8_0 # Run inference directly in the terminal: ./llama-cli -hf Sleem247/legal-ft-1-Q8_0-GGUF:Q8_0
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf Sleem247/legal-ft-1-Q8_0-GGUF:Q8_0 # Run inference directly in the terminal: ./build/bin/llama-cli -hf Sleem247/legal-ft-1-Q8_0-GGUF:Q8_0
Use Docker
docker model run hf.co/Sleem247/legal-ft-1-Q8_0-GGUF:Q8_0
- LM Studio
- Jan
- Ollama
How to use Sleem247/legal-ft-1-Q8_0-GGUF with Ollama:
ollama run hf.co/Sleem247/legal-ft-1-Q8_0-GGUF:Q8_0
- Unsloth Studio
How to use Sleem247/legal-ft-1-Q8_0-GGUF with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for Sleem247/legal-ft-1-Q8_0-GGUF to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for Sleem247/legal-ft-1-Q8_0-GGUF to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for Sleem247/legal-ft-1-Q8_0-GGUF to start chatting
- Docker Model Runner
How to use Sleem247/legal-ft-1-Q8_0-GGUF with Docker Model Runner:
docker model run hf.co/Sleem247/legal-ft-1-Q8_0-GGUF:Q8_0
- Lemonade
How to use Sleem247/legal-ft-1-Q8_0-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Sleem247/legal-ft-1-Q8_0-GGUF:Q8_0
Run and chat with the model
lemonade run user.legal-ft-1-Q8_0-GGUF-Q8_0
List all available models
lemonade list
- Atomic Chat
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:400
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
- llama-cpp
- gguf-my-repo
base_model: llm-wizard/legal-ft-1
widget:
- source_sentence: >-
Why is the use of AI systems particularly important for individuals
applying for or receiving public assistance benefits?
sentences:
- (48)
- "Another area in which the use of AI systems deserves special consideration is the access to and enjoyment of certain essential private and public services and benefits necessary for people to fully participate in society or to improve one’s standard of living. In particular, natural persons applying for or receiving essential public assistance benefits and services from public authorities namely healthcare services, social security benefits, social services providing protection in cases such as maternity, illness, industrial accidents, dependency or old age and loss of employment and social and housing assistance, are typically dependent on those benefits and services and in a\_vulnerable position in relation to the responsible authorities."
- "used for biometric verification, which includes authentication, the sole purpose of which is to confirm that a\_specific natural person is the person he or she claims to be and to confirm the identity of a\_natural person for the sole purpose of having access to a\_service, unlocking a\_device or having security access to premises. That exclusion is justified by the fact that such systems are likely to have a\_minor impact on fundamental rights of natural persons compared to the remote biometric identification systems which may be used for the processing of the biometric data of a\_large number of persons without their active involvement. In the case of ‘real-time’ systems, the capturing of the biometric data, the comparison and the"
- source_sentence: >-
How does the context ensure that existing Union law on personal data
processing remains unaffected?
sentences:
- >-
does not seek to affect the application of existing Union law governing
the processing of personal data, including the tasks and powers of the
independent supervisory authorities competent to monitor compliance with
those instruments. It also does not affect the obligations of providers
and deployers of AI systems in their role as data controllers or
processors stemming from Union or national law on the protection of
personal data in so far as the design, the development or the use of AI
systems involves the processing of personal data. It is also appropriate
to clarify that data subjects continue to enjoy all the rights and
guarantees awarded to them by such Union law, including the rights
related to solely automated individual
- "to operate without human intervention. The adaptiveness that an AI system could exhibit after deployment, refers to self-learning capabilities, allowing the system to change while in use. AI systems can be used on a\_stand-alone basis or as a\_component of a\_product, irrespective of whether the system is physically integrated into the product (embedded) or serves the functionality of the product without being integrated therein (non-embedded)."
- "requested by the European Parliament\_(6)."
- source_sentence: >-
How does the context surrounding the number 33 influence its
interpretation?
sentences:
- >-
race, sex or disabilities. In addition, the immediacy of the impact and
the limited opportunities for further checks or corrections in relation
to the use of such systems operating in real-time carry heightened risks
for the rights and freedoms of the persons concerned in the context of,
or impacted by, law enforcement activities.
- (33)
- (61)
- source_sentence: >-
What are the potential consequences of a serious disruption of critical
infrastructure as defined in Directive (EU) 2022/2557?
sentences:
- "to highly varying degrees for the practical pursuit of the localisation or identification of a\_perpetrator or suspect of the different criminal offences listed and having regard to the likely differences in the seriousness, probability and scale of the harm or possible negative consequences. An imminent threat to life or the physical safety of natural persons could also result from a\_serious disruption of critical infrastructure, as defined in Article\_2, point (4) of Directive (EU) 2022/2557 of the European Parliament and of the Council\_(19), where the disruption or destruction of such critical infrastructure would result in an imminent threat to life or the physical safety of a\_person, including through serious harm to the provision of"
- (53)
- >-
(66)
Requirements should apply to high-risk AI systems as regards risk
management, the quality and relevance of data sets used, technical
documentation and record-keeping, transparency and the provision of
information to deployers, human oversight, and robustness, accuracy and
cybersecurity. Those requirements are necessary to effectively mitigate
the risks for health, safety and fundamental rights. As no other less
trade restrictive measures are reasonably available those requirements
are not unjustified restrictions to trade.
(67)
- source_sentence: >-
What criteria determine whether an AI system used in the administration of
justice is classified as high-risk?
sentences:
- "which one or more of the following conditions are fulfilled. The first such condition should be that the AI system is intended to perform a\_narrow procedural task, such as an AI system that transforms unstructured data into structured data, an AI system that classifies incoming documents into categories or an AI system that is used to detect duplicates among a\_large number of applications. Those tasks are of such narrow and limited nature that they pose only limited risks which are not increased through the use of an AI system in a\_context that is listed as a\_high-risk use in an annex to this Regulation. The second condition should be that the task performed by the AI system is intended to improve the result of a\_previously completed human"
- "Certain AI systems intended for the administration of justice and democratic processes should be classified as high-risk, considering their potentially significant impact on democracy, the rule of law, individual freedoms as well as the right to an effective remedy and to a\_fair trial. In particular, to address the risks of potential biases, errors and opacity, it is appropriate to qualify as high-risk AI systems intended to be used by a\_judicial authority or on its behalf to assist judicial authorities in researching and interpreting facts and the law and in applying the law to a\_concrete set of facts. AI systems intended to be used by alternative dispute resolution bodies for those purposes should also be considered to be high-risk when"
- "As regards AI systems that are safety components of products, or which are themselves products, falling within the scope of certain Union harmonisation legislation listed in an annex to this Regulation, it is appropriate to classify them as high-risk under this Regulation if the product concerned undergoes the conformity assessment procedure with a\_third-party conformity assessment body pursuant to that relevant Union harmonisation legislation. In particular, such products are machinery, toys, lifts, equipment and protective systems intended for use in potentially explosive atmospheres, radio equipment, pressure equipment, recreational craft equipment, cableway installations, appliances burning gaseous fuels, medical devices, in vitro"
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
model-index:
- name: SentenceTransformer based on Snowflake/snowflake-arctic-embed-l
results:
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: Unknown
type: unknown
metrics:
- type: cosine_accuracy@1
value: 0.9583333333333334
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 1
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 1
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 1
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.9583333333333334
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.3333333333333333
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.19999999999999998
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.09999999999999999
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.9583333333333334
name: Cosine Recall@1
- type: cosine_recall@3
value: 1
name: Cosine Recall@3
- type: cosine_recall@5
value: 1
name: Cosine Recall@5
- type: cosine_recall@10
value: 1
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.9791666666666666
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.9722222222222223
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.9722222222222222
name: Cosine Map@100
Sleem247/legal-ft-1-Q8_0-GGUF
This model was converted to GGUF format from llm-wizard/legal-ft-1 using llama.cpp via the ggml.ai's GGUF-my-repo space.
Refer to the original model card for more details on the model.
Use with llama.cpp
Install llama.cpp through brew (works on Mac and Linux)
brew install llama.cpp
Invoke the llama.cpp server or the CLI.
CLI:
llama-cli --hf-repo Sleem247/legal-ft-1-Q8_0-GGUF --hf-file legal-ft-1-q8_0.gguf -p "The meaning to life and the universe is"
Server:
llama-server --hf-repo Sleem247/legal-ft-1-Q8_0-GGUF --hf-file legal-ft-1-q8_0.gguf -c 2048
Note: You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo as well.
Step 1: Clone llama.cpp from GitHub.
git clone https://github.com/ggerganov/llama.cpp
Step 2: Move into the llama.cpp folder and build it with LLAMA_CURL=1 flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
cd llama.cpp && LLAMA_CURL=1 make
Step 3: Run inference through the main binary.
./llama-cli --hf-repo Sleem247/legal-ft-1-Q8_0-GGUF --hf-file legal-ft-1-q8_0.gguf -p "The meaning to life and the universe is"
or
./llama-server --hf-repo Sleem247/legal-ft-1-Q8_0-GGUF --hf-file legal-ft-1-q8_0.gguf -c 2048