Large Language Models (LLM) - SME - REMOTE

Experience with below skills :

1. LLM

2. Langchain

3. Contextual Similarity Searching

4. Contextual Text Embedding

5. RAG

6. Knowledge of vectordb ( ChromaDB we are using ) 

9. Prompt engineering or Prompt chaining expert

------ RESPONSIBILITIES

1- LLM Capability Evaluation

 - Develop an evaluation report framework for NLP tools / LLMs for our supported languages

 - Conduct thorough audits/evaluations of client’s NLP tools / LLMs to ensure performance, high accuracy, and reliability in generated outputs.

 - Identify, report, and track issues, biases, and inconsistencies in NLP tools / LLMs outputs.

2- Performance Assessment

 - Conduct robust testing to evaluate the NLP tools / LLMs performance under varied use-case scenarios.

 - Provide comprehensive feedback and recommendations to enhance NLP tools / LLMs’ high accuracy and utility.

3- Compliance Checks

 - Ensure that the use and functioning of NLP tools / LLMs adhere to legal, regulatory, and organizational guidelines.

 - Validate that data utilised and generated by NLP tools / LLMs maintains stringent privacy and security standards.

4- Collaboration and Communication - with internal and external

 - Communicate audit findings, insights, and recommendations clearly to technical and non-technical stakeholders.

4- Ongoing Learning and Adaptation

 - Stay abreast of advancements, tools, and methodologies in NLP tools / LLMs development and auditing.

 - Continuously refine audit procedures and protocols to align with evolving technologies and standards.

5- Documentation the process

 - Maintain meticulous documentation of audit processes, findings, and ensuing actions.

Develop and disseminate reports detailing audit results and recommendations