Tree Retrievers

Summarize query.
class llama_index.indices.tree.all_leaf_retriever.TreeAllLeafRetriever(index: TreeIndex, **kwargs: Any)
   GPT all leaf retriever.
   This class builds a query-specific tree from leaf nodes to return a
   response. Using this query mode means that the tree index doesn't
   need to be built when initialized, since we rebuild the tree for
   each query.
   Parameters:
      **text_qa_template** (*Optional**[**BasePromptTemplate**]*) --
      Question-Answer Prompt (see Prompt Templates).
   get_service_context() -> Optional[ServiceContext]
      Attempts to resolve a service context. Short-circuits at
      self.service_context, self._service_context, or
      self._index.service_context.
   retrieve(str_or_query_bundle: Union[str, QueryBundle]) -> List[NodeWithScore]
      Retrieve nodes given query.
      Parameters:
         **str_or_query_bundle** (*QueryType*) -- Either a query
         string or a QueryBundle object.
Leaf query mechanism.
class llama_index.indices.tree.select_leaf_retriever.TreeSelectLeafRetriever(index: TreeIndex, query_template: Optional[BasePromptTemplate] = None, text_qa_template: Optional[BasePromptTemplate] = None, refine_template: Optional[BasePromptTemplate] = None, query_template_multiple: Optional[BasePromptTemplate] = None, child_branch_factor: int = 1, verbose: bool = False, **kwargs: Any)
   Tree select leaf retriever.
   This class traverses the index graph and searches for a leaf node
   that can best answer the query.
   Parameters:
      * **query_template** (*Optional**[**BasePromptTemplate**]*) --
        Tree Select Query Prompt (see Prompt Templates).
      * **query_template_multiple**
        (*Optional**[**BasePromptTemplate**]*) -- Tree Select Query
        Prompt (Multiple) (see Prompt Templates).
      * **child_branch_factor** (*int*) -- Number of child nodes to
        consider at each level. If child_branch_factor is 1, then the
        query will only choose one child node to traverse for any
        given parent node. If child_branch_factor is 2, then the query
        will choose two child nodes.
   get_service_context() -> Optional[ServiceContext]
      Attempts to resolve a service context. Short-circuits at
      self.service_context, self._service_context, or
      self._index.service_context.
   retrieve(str_or_query_bundle: Union[str, QueryBundle]) -> List[NodeWithScore]
      Retrieve nodes given query.
      Parameters:
         **str_or_query_bundle** (*QueryType*) -- Either a query
         string or a QueryBundle object.
llama_index.indices.tree.select_leaf_retriever.get_text_from_node(node: BaseNode, level: Optional[int] = None, verbose: bool = False) -> str
   Get text from node.
Query Tree using embedding similarity between query and node text.
class llama_index.indices.tree.select_leaf_embedding_retriever.TreeSelectLeafEmbeddingRetriever(index: TreeIndex, query_template: Optional[BasePromptTemplate] = None, text_qa_template: Optional[BasePromptTemplate] = None, refine_template: Optional[BasePromptTemplate] = None, query_template_multiple: Optional[BasePromptTemplate] = None, child_branch_factor: int = 1, verbose: bool = False, **kwargs: Any)
   Tree select leaf embedding retriever.
   This class traverses the index graph using the embedding similarity
   between the query and the node text.
   Parameters:
      * **query_template** (*Optional**[**BasePromptTemplate**]*) --
        Tree Select Query Prompt (see Prompt Templates).
      * **query_template_multiple**
        (*Optional**[**BasePromptTemplate**]*) -- Tree Select Query

Tree Retrievers

        Prompt (Multiple) (see Prompt Templates).
      * **text_qa_template** (*Optional**[**BasePromptTemplate**]*) --
        Question-Answer Prompt (see Prompt Templates).
      * **refine_template** (*Optional**[**BasePromptTemplate**]*) --
        Refinement Prompt (see Prompt Templates).
      * **child_branch_factor** (*int*) -- Number of child nodes to
        consider at each level. If child_branch_factor is 1, then the
        query will only choose one child node to traverse for any
        given parent node. If child_branch_factor is 2, then the query
        will choose two child nodes.
      * **embed_model** (*Optional**[**BaseEmbedding**]*) -- Embedding
        model to use for embedding similarity.
   get_service_context() -> Optional[ServiceContext]
      Attempts to resolve a service context. Short-circuits at
      self.service_context, self._service_context, or
      self._index.service_context.
   retrieve(str_or_query_bundle: Union[str, QueryBundle]) -> List[NodeWithScore]
      Retrieve nodes given query.
      Parameters:
         **str_or_query_bundle** (*QueryType*) -- Either a query
         string or a QueryBundle object.


Condense Question Chat Engine

class llama_index.chat_engine.condense_question.CondenseQuestionChatEngine(query_engine: BaseQueryEngine, condense_question_prompt: BasePromptTemplate, memory: BaseMemory, service_context: ServiceContext, verbose: bool = False, callback_manager: Optional[CallbackManager] = None)
   Condense Question Chat Engine.
   First generate a standalone question from conversation context and
   last message, then query the query engine for a response.
   async achat(*args: Any, **kwargs: Any) -> Any
      Async version of main chat interface.
   async astream_chat(*args: Any, **kwargs: Any) -> Any
      Async version of main chat interface.
   chat(*args: Any, **kwargs: Any) -> Any
      Main chat interface.
   property chat_history: List[ChatMessage]
      Get chat history.
   chat_repl() -> None
      Enter interactive chat REPL.
   classmethod from_defaults(query_engine: ~llama_index.indices.query.base.BaseQueryEngine, condense_question_prompt: ~typing.Optional[~llama_index.prompts.base.BasePromptTemplate] = None, chat_history: ~typing.Optional[~typing.List[~llama_index.llms.base.ChatMessage]] = None, memory: ~typing.Optional[~llama_index.memory.types.BaseMemory] = None, memory_cls: ~typing.Type[~llama_index.memory.types.BaseMemory] = <class 'llama_index.memory.chat_memory_buffer.ChatMemoryBuffer'>, service_context: ~typing.Optional[~llama_index.indices.service_context.ServiceContext] = None, verbose: bool = False, system_prompt: ~typing.Optional[str] = None, prefix_messages: ~typing.Optional[~typing.List[~llama_index.llms.base.ChatMessage]] = None, **kwargs: ~typing.Any) -> CondenseQuestionChatEngine
      Initialize a CondenseQuestionChatEngine from default parameters.
   reset() -> None
      Reset conversation state.
   stream_chat(*args: Any, **kwargs: Any) -> Any
      Stream chat interface.


Simple Chat Engine

class llama_index.chat_engine.simple.SimpleChatEngine(llm: LLM, memory: BaseMemory, prefix_messages: List[ChatMessage], callback_manager: Optional[CallbackManager] = None)
   Simple Chat Engine.
   Have a conversation with the LLM. This does not make use of a
   knowledge base.
   async achat(*args: Any, **kwargs: Any) -> Any
      Async version of main chat interface.
   async astream_chat(*args: Any, **kwargs: Any) -> Any
      Async version of main chat interface.
   chat(*args: Any, **kwargs: Any) -> Any
      Main chat interface.
   property chat_history: List[ChatMessage]
      Get chat history.
   chat_repl() -> None
      Enter interactive chat REPL.
   classmethod from_defaults(service_context: ~typing.Optional[~llama_index.indices.service_context.ServiceContext] = None, chat_history: ~typing.Optional[~typing.List[~llama_index.llms.base.ChatMessage]] = None, memory: ~typing.Optional[~llama_index.memory.types.BaseMemory] = None, memory_cls: ~typing.Type[~llama_index.memory.types.BaseMemory] = <class 'llama_index.memory.chat_memory_buffer.ChatMemoryBuffer'>, system_prompt: ~typing.Optional[str] = None, prefix_messages: ~typing.Optional[~typing.List[~llama_index.llms.base.ChatMessage]] = None, **kwargs: ~typing.Any) -> SimpleChatEngine
      Initialize a SimpleChatEngine from default parameters.
   reset() -> None
      Reset conversation state.
   stream_chat(*args: Any, **kwargs: Any) -> Any
      Stream chat interface.


Retriever Query Engine

class llama_index.query_engine.retriever_query_engine.RetrieverQueryEngine(retriever: BaseRetriever, response_synthesizer: Optional[BaseSynthesizer] = None, node_postprocessors: Optional[List[BaseNodePostprocessor]] = None, callback_manager: Optional[CallbackManager] = None)
   Retriever query engine.
   Parameters:
      * **retriever** (*BaseRetriever*) -- A retriever object.
      * **response_synthesizer** (*Optional**[**BaseSynthesizer**]*)
        -- A BaseSynthesizer object.
      * **callback_manager** (*Optional**[**CallbackManager**]*) -- A
        callback manager.
   classmethod from_args(retriever: BaseRetriever, response_synthesizer: Optional[BaseSynthesizer] = None, service_context: Optional[ServiceContext] = None, node_postprocessors: Optional[List[BaseNodePostprocessor]] = None, response_mode: ResponseMode = ResponseMode.COMPACT, text_qa_template: Optional[BasePromptTemplate] = None, refine_template: Optional[BasePromptTemplate] = None, summary_template: Optional[BasePromptTemplate] = None, simple_template: Optional[BasePromptTemplate] = None, output_cls: Optional[BaseModel] = None, use_async: bool = False, streaming: bool = False, **kwargs: Any) -> RetrieverQueryEngine
      Initialize a RetrieverQueryEngine object.".
      Parameters:
         * **retriever** (*BaseRetriever*) -- A retriever object.
         * **service_context** (*Optional**[**ServiceContext**]*) -- A
           ServiceContext object.
         * **node_postprocessors**
           (*Optional**[**List**[**BaseNodePostprocessor**]**]*) -- A
           list of node postprocessors.
         * **verbose** (*bool*) -- Whether to print out debug info.
         * **response_mode** (*ResponseMode*) -- A ResponseMode
           object.
         * **text_qa_template** (*Optional**[**BasePromptTemplate**]*)
           -- A BasePromptTemplate object.
         * **refine_template** (*Optional**[**BasePromptTemplate**]*)
           -- A BasePromptTemplate object.
         * **simple_template** (*Optional**[**BasePromptTemplate**]*)
           -- A BasePromptTemplate object.
         * **use_async** (*bool*) -- Whether to use async.
         * **streaming** (*bool*) -- Whether to use streaming.
         * **optimizer** (*Optional**[**BaseTokenUsageOptimizer**]*)
           -- A BaseTokenUsageOptimizer object.
   property retriever: BaseRetriever
      Get the retriever object.


Retriever Router Query Engine


Graph Query Engine

class llama_index.query_engine.graph_query_engine.ComposableGraphQueryEngine(graph: ComposableGraph, custom_query_engines: Optional[Dict[str, BaseQueryEngine]] = None, recursive: bool = True, **kwargs: Any)
   Composable graph query engine.
   This query engine can operate over a ComposableGraph. It can take
   in custom query engines for its sub-indices.
   Parameters:
      * **graph** (*ComposableGraph*) -- A ComposableGraph object.
      * **custom_query_engines** (*Optional**[**Dict**[**str**,
        **BaseQueryEngine**]**]*) -- A dictionary of custom query
        engines.
      * **recursive** (*bool*) -- Whether to recursively query the
        graph.
      * ****kwargs** -- additional arguments to be passed to the
        underlying index query engine.


Knowledge Graph Query Engine

Knowledge Graph Query Engine.
class llama_index.query_engine.knowledge_graph_query_engine.KnowledgeGraphQueryEngine(service_context: Optional[ServiceContext] = None, storage_context: Optional[StorageContext] = None, graph_query_synthesis_prompt: Optional[BasePromptTemplate] = None, graph_response_answer_prompt: Optional[BasePromptTemplate] = None, refresh_schema: bool = False, verbose: bool = False, response_synthesizer: Optional[BaseSynthesizer] = None, **kwargs: Any)
   Knowledge graph query engine.
   Query engine to call a knowledge graph.
   Parameters:
      * **service_context** (*Optional**[**ServiceContext**]*) -- A
        service context to use.
      * **storage_context** (*Optional**[**StorageContext**]*) -- A
        storage context to use.
      * **refresh_schema** (*bool*) -- Whether to refresh the schema.
      * **verbose** (*bool*) -- Whether to print intermediate results.
      * **response_synthesizer** (*Optional**[**BaseSynthesizer**]*)
        -- A BaseSynthesizer object.
      * ****kwargs** -- Additional keyword arguments.
   async agenerate_query(query_str: str) -> str
      Generate a Graph Store Query from a query bundle.
   generate_query(query_str: str) -> str
      Generate a Graph Store Query from a query bundle.


SQL Query Engine

Default query for SQLStructStoreIndex.
class llama_index.indices.struct_store.sql_query.BaseSQLTableQueryEngine(sql_database: SQLDatabase, text_to_sql_prompt: Optional[BasePromptTemplate] = None, context_query_kwargs: Optional[dict] = None, synthesize_response: bool = True, response_synthesis_prompt: Optional[BasePromptTemplate] = None, service_context: Optional[ServiceContext] = None, **kwargs: Any)
   property service_context: ServiceContext
      Get service context.
llama_index.indices.struct_store.sql_query.GPTNLStructStoreQueryEngine
   alias of "NLStructStoreQueryEngine"
llama_index.indices.struct_store.sql_query.GPTSQLStructStoreQueryEngine
   alias of "SQLStructStoreQueryEngine"
class llama_index.indices.struct_store.sql_query.NLSQLTableQueryEngine(sql_database: SQLDatabase, text_to_sql_prompt: Optional[BasePromptTemplate] = None, context_query_kwargs: Optional[dict] = None, synthesize_response: bool = True, response_synthesis_prompt: Optional[BasePromptTemplate] = None, tables: Optional[Union[List[str], List[Table]]] = None, service_context: Optional[ServiceContext] = None, **kwargs: Any)
   Natural language SQL Table query engine.
   Read NLStructStoreQueryEngine's docstring for more info on NL SQL.
   property service_context: ServiceContext
      Get service context.
class llama_index.indices.struct_store.sql_query.NLStructStoreQueryEngine(index: SQLStructStoreIndex, text_to_sql_prompt: Optional[BasePromptTemplate] = None, context_query_kwargs: Optional[dict] = None, synthesize_response: bool = True, response_synthesis_prompt: Optional[BasePromptTemplate] = None, **kwargs: Any)
   GPT natural language query engine over a structured database.
   NOTE: deprecated in favor of SQLTableRetriever, kept for backward
   compatibility.
   Given a natural language query, we will extract the query to SQL.
   Runs raw SQL over a SQLStructStoreIndex. No LLM calls are made
   during the SQL execution.
   NOTE: this query cannot work with composed indices - if the index
   contains subindices, those subindices will not be queried.
   Parameters:
      * **index** (*SQLStructStoreIndex*) -- A SQL Struct Store Index
      * **text_to_sql_prompt** (*Optional**[**BasePromptTemplate**]*)
        -- A Text to SQL BasePromptTemplate to use for the query.
        Defaults to DEFAULT_TEXT_TO_SQL_PROMPT.
      * **context_query_kwargs** (*Optional**[**dict**]*) -- Keyword
        arguments for the context query. Defaults to {}.
      * **synthesize_response** (*bool*) -- Whether to synthesize a
        response from the query results. Defaults to True.
      * **response_synthesis_prompt**
        (*Optional**[**BasePromptTemplate**]*) -- A Response Synthesis
        BasePromptTemplate to use for the query. Defaults to
        DEFAULT_RESPONSE_SYNTHESIS_PROMPT.
   property service_context: ServiceContext
      Get service context.
class llama_index.indices.struct_store.sql_query.PGVectorSQLQueryEngine(sql_database: SQLDatabase, text_to_sql_prompt: Optional[BasePromptTemplate] = None, context_query_kwargs: Optional[dict] = None, synthesize_response: bool = True, response_synthesis_prompt: Optional[BasePromptTemplate] = None, tables: Optional[Union[List[str], List[Table]]] = None, service_context: Optional[ServiceContext] = None, **kwargs: Any)
   PGvector SQL query engine.
   A modified version of the normal text-to-SQL query engine because
   we can infer embedding vectors in the sql query.
   NOTE: this is a beta feature
   property service_context: ServiceContext
      Get service context.
class llama_index.indices.struct_store.sql_query.SQLStructStoreQueryEngine(index: SQLStructStoreIndex, sql_context_container: Optional[SQLContextContainerBuilder] = None, **kwargs: Any)

SQL Query Engine

   GPT SQL query engine over a structured database.
   NOTE: deprecated in favor of SQLTableRetriever, kept for backward
   compatibility.
   Runs raw SQL over a SQLStructStoreIndex. No LLM calls are made
   here. NOTE: this query cannot work with composed indices - if the
   index contains subindices, those subindices will not be queried.
class llama_index.indices.struct_store.sql_query.SQLTableRetrieverQueryEngine(sql_database: SQLDatabase, table_retriever: ObjectRetriever[SQLTableSchema], text_to_sql_prompt: Optional[BasePromptTemplate] = None, context_query_kwargs: Optional[dict] = None, synthesize_response: bool = True, response_synthesis_prompt: Optional[BasePromptTemplate] = None, service_context: Optional[ServiceContext] = None, context_str_prefix: Optional[str] = None, **kwargs: Any)
   SQL Table retriever query engine.
   property service_context: ServiceContext
      Get service context.


Router Query Engine

class llama_index.query_engine.router_query_engine.RetrieverRouterQueryEngine(retriever: BaseRetriever, node_to_query_engine_fn: Callable, callback_manager: Optional[CallbackManager] = None)
   Retriever-based router query engine.
   NOTE: this is deprecated, please use our new
   ToolRetrieverRouterQueryEngine
   Use a retriever to select a set of Nodes. Each node will be
   converted into a ToolMetadata object, and also used to retrieve a
   query engine, to form a QueryEngineTool.
   NOTE: this is a beta feature. We are figuring out the right
   interface between the retriever and query engine.
   Parameters:
      * **selector** (*BaseSelector*) -- A selector that chooses one
        out of many options based on each candidate's metadata and
        query.
      * **query_engine_tools** (*Sequence**[**QueryEngineTool**]*) --
        A sequence of candidate query engines. They must be wrapped as
        tools to expose metadata to the selector.
      * **callback_manager** (*Optional**[**CallbackManager**]*) -- A
        callback manager.
class llama_index.query_engine.router_query_engine.RouterQueryEngine(selector: BaseSelector, query_engine_tools: Sequence[QueryEngineTool], service_context: Optional[ServiceContext] = None, summarizer: Optional[TreeSummarize] = None)
   Router query engine.
   Selects one out of several candidate query engines to execute a
   query.
   Parameters:
      * **selector** (*BaseSelector*) -- A selector that chooses one
        out of many options based on each candidate's metadata and
        query.
      * **query_engine_tools** (*Sequence**[**QueryEngineTool**]*) --
        A sequence of candidate query engines. They must be wrapped as
        tools to expose metadata to the selector.
      * **service_context** (*Optional**[**ServiceContext**]*) -- A
        service context.
      * **summarizer** (*Optional**[**TreeSummarize**]*) -- Tree
        summarizer to summarize sub-results.
class llama_index.query_engine.router_query_engine.ToolRetrieverRouterQueryEngine(retriever: ObjectRetriever[QueryEngineTool], service_context: Optional[ServiceContext] = None, summarizer: Optional[TreeSummarize] = None)
   Tool Retriever router query engine.
   Selects a set of candidate query engines to execute a query.
   Parameters:
      * **retriever** (*ObjectRetriever*) -- A retriever that
        retrieves a set of query engine tools.
      * **service_context** (*Optional**[**ServiceContext**]*) -- A
        service context.
      * **summarizer** (*Optional**[**TreeSummarize**]*) -- Tree
        summarizer to summarize sub-results.


Sub Question Query Engine

pydantic model llama_index.query_engine.sub_question_query_engine.SubQuestionAnswerPair
   Pair of the sub question and optionally its answer (if its been
   answered yet).
      {
         "title": "SubQuestionAnswerPair",
         "description": "Pair of the sub question and optionally its answer (if its been answered yet).",
         "type": "object",
         "properties": {
            "sub_q": {
               "$ref": "#/definitions/SubQuestion"
            },
            "answer": {
               "title": "Answer",
               "type": "string"
            },
            "sources": {
               "title": "Sources",
               "type": "array",
               "items": {
                  "$ref": "#/definitions/NodeWithScore"
               }
            }
         },
         "required": [
            "sub_q"
         ],
         "definitions": {
            "SubQuestion": {
               "title": "SubQuestion",
               "type": "object",
               "properties": {
                  "sub_question": {
                     "title": "Sub Question",
                     "type": "string"
                  },
                  "tool_name": {
                     "title": "Tool Name",
                     "type": "string"
                  }
               },
               "required": [
                  "sub_question",
                  "tool_name"
               ]
            },
            "ObjectType": {
               "title": "ObjectType",
               "description": "An enumeration.",
               "enum": [
                  "1",
                  "2",
                  "3",
                  "4"
               ],
               "type": "string"
            },
            "RelatedNodeInfo": {
               "title": "RelatedNodeInfo",
               "description": "Base component object to capture class names.",
               "type": "object",
               "properties": {
                  "node_id": {
                     "title": "Node Id",
                     "type": "string"
                  },
                  "node_type": {
                     "$ref": "#/definitions/ObjectType"
                  },
                  "metadata": {
                     "title": "Metadata",
                     "type": "object"
                  },
                  "hash": {
                     "title": "Hash",
                     "type": "string"
                  }
               },
               "required": [
                  "node_id"
               ]
            },
            "BaseNode": {
               "title": "BaseNode",
               "description": "Base node Object.\n\nGeneric abstract interface for retrievable nodes",
               "type": "object",
               "properties": {
                  "id_": {
                     "title": "Id ",
                     "description": "Unique ID of the node.",
                     "type": "string"
                  },
                  "embedding": {
                     "title": "Embedding",
                     "description": "Embedding of the node.",
                     "type": "array",
                     "items": {
                        "type": "number"
                     }
                  },
                  "extra_info": {
                     "title": "Extra Info",
                     "description": "A flat dictionary of metadata fields",
                     "type": "object"
                  },
                  "excluded_embed_metadata_keys": {
                     "title": "Excluded Embed Metadata Keys",
                     "description": "Metadata keys that are excluded from text for the embed model.",
                     "type": "array",
                     "items": {
                        "type": "string"
                     }
                  },
                  "excluded_llm_metadata_keys": {
                     "title": "Excluded Llm Metadata Keys",
                     "description": "Metadata keys that are excluded from text for the LLM.",
                     "type": "array",
                     "items": {
                        "type": "string"
                     }
                  },
                  "relationships": {
                     "title": "Relationships",
                     "description": "A mapping of relationships to other node information.",
                     "type": "object",
                     "additionalProperties": {
                        "anyOf": [
                           {
                              "$ref": "#/definitions/RelatedNodeInfo"

Sub Question Query Engine

                           },
                           {
                              "type": "array",
                              "items": {
                                 "$ref": "#/definitions/RelatedNodeInfo"
                              }
                           }
                        ]
                     }
                  },
                  "hash": {
                     "title": "Hash",
                     "description": "Hash of the node content.",
                     "default": "",
                     "type": "string"
                  }
               }
            },
            "NodeWithScore": {
               "title": "NodeWithScore",
               "description": "Base component object to capture class names.",
               "type": "object",
               "properties": {
                  "node": {
                     "$ref": "#/definitions/BaseNode"
                  },
                  "score": {
                     "title": "Score",
                     "type": "number"
                  }
               },
               "required": [
                  "node"
               ]
            }
         }
      }
   Fields:
      * "answer (Optional[str])"
      * "sources (List[llama_index.schema.NodeWithScore])"
      * "sub_q (llama_index.question_gen.types.SubQuestion)"
   field answer: Optional[str] = None
   field sources: List[NodeWithScore] [Optional]
   field sub_q: SubQuestion [Required]
   classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) -> Model
      Creates a new model setting __dict__ and __fields_set__ from
      trusted or pre-validated data. Default values are respected, but
      no other validation is performed. Behaves as if *Config.extra =
      'allow'* was set since it adds all passed values
   copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) -> Model
      Duplicate a model, optionally choose which fields to include,
      exclude and change.
      Parameters:
         * **include** -- fields to include in new model
         * **exclude** -- fields to exclude from new model, as with
           values this takes precedence over include
         * **update** -- values to change/add in the new model. Note:
           the data is not validated before creating the new model:
           you should trust this data
         * **deep** -- set to *True* to make a deep copy of the model
      Returns:
         new model instance
   dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) -> DictStrAny
      Generate a dictionary representation of the model, optionally
      specifying which fields to include or exclude.
   classmethod from_orm(obj: Any) -> Model
   json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) -> unicode
      Generate a JSON representation of the model, *include* and
      *exclude* arguments as per *dict()*.
      *encoder* is an optional function to supply as *default* to
      json.dumps(), other arguments as per *json.dumps()*.

Sub Question Query Engine

   classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   classmethod parse_obj(obj: Any) -> Model
   classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') -> DictStrAny
   classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) -> unicode
   classmethod update_forward_refs(**localns: Any) -> None
      Try to update ForwardRefs on fields based on this Model,
      globalns and localns.
   classmethod validate(value: Any) -> Model
class llama_index.query_engine.sub_question_query_engine.SubQuestionQueryEngine(question_gen: BaseQuestionGenerator, response_synthesizer: BaseSynthesizer, query_engine_tools: Sequence[QueryEngineTool], callback_manager: Optional[CallbackManager] = None, verbose: bool = True, use_async: bool = False)
   Sub question query engine.
   A query engine that breaks down a complex query (e.g. compare and
   contrast) into
      many sub questions and their target query engine for execution.
      After executing all sub questions, all responses are gathered
      and sent to response synthesizer to produce the final response.
   Parameters:
      * **question_gen** (*BaseQuestionGenerator*) -- A module for
        generating sub questions given a complex question and tools.
      * **response_synthesizer** (*BaseSynthesizer*) -- A response
        synthesizer for generating the final response
      * **query_engine_tools** (*Sequence**[**QueryEngineTool**]*) --
        Tools to answer the sub questions.
      * **verbose** (*bool*) -- whether to print intermediate
        questions and answers. Defaults to True
      * **use_async** (*bool*) -- whether to execute the sub questions
        with asyncio. Defaults to True


Transform Query Engine

class llama_index.query_engine.transform_query_engine.TransformQueryEngine(query_engine: BaseQueryEngine, query_transform: BaseQueryTransform, transform_metadata: Optional[dict] = None, callback_manager: Optional[CallbackManager] = None)
   Transform query engine.
   Applies a query transform to a query bundle before passing
      it to a query engine.
   Parameters:
      * **query_engine** (*BaseQueryEngine*) -- A query engine object.
      * **query_transform** (*BaseQueryTransform*) -- A query
        transform object.
      * **transform_metadata** (*Optional**[**dict**]*) -- metadata to
        pass to the query transform.
      * **callback_manager** (*Optional**[**CallbackManager**]*) -- A
        callback manager.


Pandas Query Engine

Default query for PandasIndex.
WARNING: This tool provides the Agent access to the *eval* function.
Arbitrary code execution is possible on the machine running this tool.
This tool is not recommended to be used in a production setting, and
would require heavy sandboxing or virtual machines
llama_index.query_engine.pandas_query_engine.GPTNLPandasQueryEngine
   alias of "PandasQueryEngine"
llama_index.query_engine.pandas_query_engine.NLPandasQueryEngine
   alias of "PandasQueryEngine"
class llama_index.query_engine.pandas_query_engine.PandasQueryEngine(df: DataFrame, instruction_str: Optional[str] = None, output_processor: Optional[Callable] = None, pandas_prompt: Optional[BasePromptTemplate] = None, output_kwargs: Optional[dict] = None, head: int = 5, verbose: bool = False, service_context: Optional[ServiceContext] = None, **kwargs: Any)
   GPT Pandas query.
   Convert natural language to Pandas python code.
   WARNING: This tool provides the Agent access to the *eval*
   function. Arbitrary code execution is possible on the machine
   running this tool. This tool is not recommended to be used in a
   production setting, and would require heavy sandboxing or virtual
   machines
   Parameters:
      * **df** (*pd.DataFrame*) -- Pandas dataframe to use.
      * **instruction_str** (*Optional**[**str**]*) -- Instruction
        string to use.
      * **output_processor** (*Optional**[**Callable**[**[**str**]**,
        **str**]**]*) -- Output processor. A callable that takes in
        the output string, pandas DataFrame, and any output kwargs and
        returns a string.
      * **pandas_prompt** (*Optional**[**BasePromptTemplate**]*) --
        Pandas prompt to use.
      * **head** (*int*) -- Number of rows to show in the table
        context.
llama_index.query_engine.pandas_query_engine.default_output_processor(output: str, df: DataFrame, **output_kwargs: Any) -> str
   Process outputs in a default manner.


Citation Query Engine

class llama_index.query_engine.citation_query_engine.CitationQueryEngine(retriever: BaseRetriever, response_synthesizer: Optional[BaseSynthesizer] = None, citation_chunk_size: int = 512, citation_chunk_overlap: int = 20, text_splitter: Optional[TextSplitter] = None, node_postprocessors: Optional[List[BaseNodePostprocessor]] = None, callback_manager: Optional[CallbackManager] = None)
   Citation query engine.
   Parameters:
      * **retriever** (*BaseRetriever*) -- A retriever object.
      * **response_synthesizer** (*Optional**[**BaseSynthesizer**]*)
        -- A BaseSynthesizer object.
      * **citation_chunk_size** (*int*) -- Size of citation chunks,
        default=512. Useful for controlling granularity of sources.
      * **citation_chunk_overlap** (*int*) -- Overlap of citation
        nodes, default=20.
      * **text_splitter** (*Optional**[**TextSplitterType**]*) -- A
        text splitter for creating citation source nodes. Default is a
        SentenceSplitter.
      * **callback_manager** (*Optional**[**CallbackManager**]*) -- A
        callback manager.
   classmethod from_args(index: ~llama_index.indices.base.BaseIndex, response_synthesizer: ~typing.Optional[~llama_index.response_synthesizers.base.BaseSynthesizer] = None, citation_chunk_size: int = 512, citation_chunk_overlap: int = 20, text_splitter: ~typing.Optional[~llama_index.text_splitter.types.TextSplitter] = None, citation_qa_template: ~llama_index.prompts.base.BasePromptTemplate = PromptTemplate(metadata={'prompt_type': <PromptType.CUSTOM: 'custom'>}, template_vars=['context_str', 'query_str'], kwargs={}, output_parser=None, template="Please provide an answer based solely on the provided sources. When referencing information from a source, cite the appropriate source(s) using their corresponding numbers. Every answer should include at least one source citation. Only cite a source when you are explicitly referencing it. If none of the sources are helpful, you should indicate that. For example:\nSource 1:\nThe sky is red in the evening and blue in the morning.\nSource 2:\nWater is wet when the sky is red.\nQuery: When is water wet?\nAnswer: Water will be wet when the sky is red [2], which occurs in the evening [1].\nNow it's your turn. Below are several numbered sources of information:\n------\n{context_str}\n------\nQuery: {query_str}\nAnswer: "), citation_refine_template: ~llama_index.prompts.base.BasePromptTemplate = PromptTemplate(metadata={'prompt_type': <PromptType.CUSTOM: 'custom'>}, template_vars=['existing_answer', 'context_msg', 'query_str'], kwargs={}, output_parser=None, template="Please provide an answer based solely on the provided sources. When referencing information from a source, cite the appropriate source(s) using their corresponding numbers. Every answer should include at least one source citation. Only cite a source when you are explicitly referencing it. If none of the sources are helpful, you should indicate that. For example:\nSource 1:\nThe sky is red in the evening and blue in the morning.\nSource 2:\nWater is wet when the sky is red.\nQuery: When is water wet?\nAnswer: Water will be wet when the sky is red [2], which occurs in the evening [1].\nNow it's your turn. We have provided an existing answer: {existing_answer}Below are several numbered sources of information. Use them to refine the existing answer. If the provided sources are not helpful, you will repeat the existing answer.\nBegin refining!\n------\n{context_msg}\n------\nQuery: {query_str}\nAnswer: "), retriever: ~typing.Optional[~llama_index.indices.base_retriever.BaseRetriever] = None, node_postprocessors: ~typing.Optional[~typing.List[~llama_index.indices.postprocessor.types.BaseNodePostprocessor]] = None, response_mode: ~llama_index.response_synthesizers.type.ResponseMode = ResponseMode.COMPACT, use_async: bool = False, streaming: bool = False, **kwargs: ~typing.Any) -> CitationQueryEngine

Citation Query Engine

      Initialize a CitationQueryEngine object.".
      Parameters:
         * **index** -- (BastGPTIndex): index to use for querying
         * **citation_chunk_size** (*int*) -- Size of citation chunks,
           default=512. Useful for controlling granularity of sources.
         * **citation_chunk_overlap** (*int*) -- Overlap of citation
           nodes, default=20.
         * **text_splitter** (*Optional**[**TextSplitter**]*) -- A
           text splitter for creating citation source nodes. Default
           is a SentenceSplitter.
         * **citation_qa_template** (*BasePromptTemplate*) -- Template
           for initial citation QA
         * **citation_refine_template** (*BasePromptTemplate*) --
           Template for citation refinement.
         * **retriever** (*BaseRetriever*) -- A retriever object.
         * **service_context** (*Optional**[**ServiceContext**]*) -- A
           ServiceContext object.
         * **node_postprocessors**
           (*Optional**[**List**[**BaseNodePostprocessor**]**]*) -- A
           list of node postprocessors.
         * **verbose** (*bool*) -- Whether to print out debug info.
         * **response_mode** (*ResponseMode*) -- A ResponseMode
           object.
         * **use_async** (*bool*) -- Whether to use async.
         * **streaming** (*bool*) -- Whether to use streaming.
         * **optimizer** (*Optional**[**BaseTokenUsageOptimizer**]*)
           -- A BaseTokenUsageOptimizer object.
   property retriever: BaseRetriever
      Get the retriever object.


Multistep Query Engine

class llama_index.query_engine.multistep_query_engine.MultiStepQueryEngine(query_engine: BaseQueryEngine, query_transform: StepDecomposeQueryTransform, response_synthesizer: Optional[BaseSynthesizer] = None, num_steps: Optional[int] = 3, early_stopping: bool = True, index_summary: str = 'None', stop_fn: Optional[Callable[[Dict], bool]] = None)
   Multi-step query engine.
   This query engine can operate over an existing base query engine,
   along with the multi-step query transform.
   Parameters:
      * **query_engine** (*BaseQueryEngine*) -- A BaseQueryEngine
        object.
      * **query_transform** (*StepDecomposeQueryTransform*) -- A
        StepDecomposeQueryTransform object.
      * **response_synthesizer** (*Optional**[**BaseSynthesizer**]*)
        -- A BaseSynthesizer object.
      * **num_steps** (*Optional**[**int**]*) -- Number of steps to
        run the multi-step query.
      * **early_stopping** (*bool*) -- Whether to stop early if the
        stop function returns True.
      * **index_summary** (*str*) -- A string summary of the index.
      * **stop_fn** (*Optional**[**Callable**[**[**Dict**]**,
        **bool**]**]*) -- A stop function that takes in a dictionary
        of information and returns a boolean.
llama_index.query_engine.multistep_query_engine.default_stop_fn(stop_dict: Dict) -> bool
   Stop function for multi-step query combiner.


SQL Join Query Engine

SQL Join query engine.
class llama_index.query_engine.sql_join_query_engine.SQLAugmentQueryTransform(llm_predictor: Optional[BaseLLMPredictor] = None, sql_augment_transform_prompt: Optional[BasePromptTemplate] = None, check_stop_parser: Optional[Callable[[QueryBundle], bool]] = None)
   SQL Augment Query Transform.
   This query transform will transform the query into a more specific
   query after augmenting with SQL results.
   Parameters:
      * **llm_predictor** (*LLMPredictor*) -- LLM predictor to use for
        query transformation.
      * **sql_augment_transform_prompt** (*BasePromptTemplate*) --
        PromptTemplate to use for query transformation.
      * **check_stop_parser** (*Optional**[**Callable**[**[**str**]**,
        **bool**]**]*) -- Check stop function.
   check_stop(query_bundle: QueryBundle) -> bool
      Check if query indicates stop.
   run(query_bundle_or_str: Union[str, QueryBundle], metadata: Optional[Dict] = None) -> QueryBundle
      Run query transform.
class llama_index.query_engine.sql_join_query_engine.SQLJoinQueryEngine(sql_query_tool: QueryEngineTool, other_query_tool: QueryEngineTool, selector: Optional[Union[LLMSingleSelector, PydanticSingleSelector]] = None, service_context: Optional[ServiceContext] = None, sql_join_synthesis_prompt: Optional[BasePromptTemplate] = None, sql_augment_query_transform: Optional[SQLAugmentQueryTransform] = None, use_sql_join_synthesis: bool = True, callback_manager: Optional[CallbackManager] = None, verbose: bool = True)
   SQL Join Query Engine.
   This query engine can "Join" a SQL database results with another
   query engine. It can decide it needs to query the SQL database or
   the other query engine. If it decides to query the SQL database, it
   will first query the SQL database, whether to augment information
   with retrieved results from the other query engine.
   Parameters:
      * **sql_query_tool** (*QueryEngineTool*) -- Query engine tool
        for SQL database. other_query_tool (QueryEngineTool): Other
        query engine tool.
      * **selector** (*Optional**[**Union**[**LLMSingleSelector**,
        **PydanticSingleSelector**]**]*) -- Selector to use.
      * **service_context** (*Optional**[**ServiceContext**]*) --
        Service context to use.
      * **sql_join_synthesis_prompt**
        (*Optional**[**BasePromptTemplate**]*) -- PromptTemplate to
        use for SQL join synthesis.
      * **sql_augment_query_transform**
        (*Optional**[**SQLAugmentQueryTransform**]*) -- Query
        transform to use for SQL augmentation.
      * **use_sql_join_synthesis** (*bool*) -- Whether to use SQL join
        synthesis.
      * **callback_manager** (*Optional**[**CallbackManager**]*) --
        Callback manager to use.
      * **verbose** (*bool*) -- Whether to print intermediate results.


Flare Query Engine

Query engines based on the FLARE paper.
Active Retrieval Augmented Generation.
class llama_index.query_engine.flare.base.FLAREInstructQueryEngine(query_engine: BaseQueryEngine, service_context: Optional[ServiceContext] = None, instruct_prompt: Optional[BasePromptTemplate] = None, lookahead_answer_inserter: Optional[BaseLookaheadAnswerInserter] = None, done_output_parser: Optional[IsDoneOutputParser] = None, query_task_output_parser: Optional[QueryTaskOutputParser] = None, max_iterations: int = 10, max_lookahead_query_tasks: int = 1, callback_manager: Optional[CallbackManager] = None, verbose: bool = False)
   FLARE Instruct query engine.
   This is the version of FLARE that uses retrieval-encouraging
   instructions.
   NOTE: this is a beta feature. Interfaces might change, and it might
   not always give correct answers.
   Parameters:
      * **query_engine** (*BaseQueryEngine*) -- query engine to use
      * **service_context** (*Optional**[**ServiceContext**]*) --
        service context. Defaults to None.
      * **instruct_prompt** (*Optional**[**PromptTemplate**]*) --
        instruct prompt. Defaults to None.
      * **lookahead_answer_inserter**
        (*Optional**[**BaseLookaheadAnswerInserter**]*) -- lookahead
        answer inserter. Defaults to None.
      * **done_output_parser** (*Optional**[**IsDoneOutputParser**]*)
        -- done output parser. Defaults to None.
      * **query_task_output_parser**
        (*Optional**[**QueryTaskOutputParser**]*) -- query task output
        parser. Defaults to None.
      * **max_iterations** (*int*) -- max iterations. Defaults to 10.
      * **max_lookahead_query_tasks** (*int*) -- max lookahead query
        tasks. Defaults to 1.
      * **callback_manager** (*Optional**[**CallbackManager**]*) --
        callback manager. Defaults to None.
      * **verbose** (*bool*) -- give verbose outputs. Defaults to
        False.


from llama_index import (

       SimpleDirectoryReader,
       VectorStoreIndex,
       download_loader,
       RAKEKeywordTableIndex,
   )
Set service context to enable streaming
   from llama_index import LLMPredictor, ServiceContext
   from llama_index.llms import OpenAI
   service_context = ServiceContext.from_defaults(
       llm=OpenAI(temperature=0, model="text-davinci-003")
   )
Load document and build index
   reader = SimpleDirectoryReader(input_files=["../data/10k/lyft_2021.pdf"])
   data = reader.load_data()
   index = VectorStoreIndex.from_documents(data, service_context=service_context)
   query_engine = index.as_query_engine(streaming=True, similarity_top_k=3)
Stream response with page citation
   response = query_engine.query(
       "What was the impact of COVID? Show statements in bullet form and show page reference after each statement."
   )
   response.print_response_stream()
   • The ongoing COVID-19 pandemic continues to impact communities in the United States, Canada and globally (page 6).
   • The pandemic and related responses caused decreased demand for our platform leading to decreased revenues as well as decreased earning opportunities for drivers on our platform (page 6).
   • Our business continues to be impacted by the COVID-19 pandemic (page 6).
   • The exact timing and pace of the recovery remain uncertain (page 6).
   • The extent to which our operations will continue to be impacted by the pandemic will depend largely on future developments, which are highly uncertain and cannot be accurately predicted (page 6).
   • An increase in cases due to variants of the virus has caused many businesses to delay employees returning to the office (page 6).
   • We anticipate that continued social distancing, altered consumer behavior, reduced travel and commuting, and expected corporate cost cutting will be significant challenges for us (page 6).
   • We have adopted multiple measures, including, but not limited, to establishing new health and safety requirements for ridesharing and updating workplace policies (page 6).
   • We have had to take certain cost-cutting measures, including lay-offs, furloughs and salary reductions, which may have adversely affect employee morale, our culture and our ability to attract and retain employees (page 18).
   • The ultimate impact of the COVID-19 pandemic on our users, customers, employees, business, operations and financial performance depends on many factors that are not within our control (page 18).
Inspect source nodes
   for node in response.source_nodes:
       print("-----")
       text_fmt = node.node.get_content().strip().replace("\n", " ")[:1000]
       print(f"Text:\t {text_fmt} ...")
       print(f"Metadata:\t {node.node.metadata}")
       print(f"Score:\t {node.score:.3f}")
   -----
   Text:	 Impact of COVID-19 to our BusinessThe  ongoing  COVID-19  pandemic  continues  to  impact  communities  in  the  United  States,  Canada  and  globally.  Since  the  pandemic  began  in  March  2020,governments  and  private  businesses  -  at  the  recommendation  of  public  health  officials  -  have  enacted  precautions  to  mitigate  the  spread  of  the  virus,  including  travelrestrictions  and  social  distancing  measures  in  many  regions  of  the  United  States  and  Canada,  and  many  enterprises  have  instituted  and  maintained  work  from  homeprograms and limited  the number of employees on site. Beginning in the middle of March 2020, the pandemic and these related responses caused decreased demand for ourplatform  leading to decreased revenues as well as decreased earning opportunities for drivers on our platform. Our business continues to be impacted by the COVID-19pandemic. Although  we have seen some signs of demand improving, particularly compared to the dema ...

from llama_index import (

   Metadata:	 {'page_label': '6', 'file_name': 'lyft_2021.pdf'}
   Score:	 0.821
   -----
   Text:	 will  continue  to  be  impacted  by  the  pandemic  will  depend  largely  on  future  developments,  which  are  highly  uncertain  and  cannot  beaccurately predicted,  including new information which may emerge concerning COVID-19 variants and the severity of the pandemic and actions by government authoritiesand  private businesses to contain the pandemic or recover from its impact, among other things. For example, an increase in cases due to variants of the virus has causedmany  businesses to delay employees returning to the office. Even as travel restrictions and shelter-in-place orders are modified or lifted, we anticipate that continued socialdistancing, altered consu mer behavior, reduced travel and commuting, and expected corporate cost cutting will be significant challenges for us. The strength and duration ofthese challenges cannot b e presently estimated.In  response to the COVID-19 pandemic, we have adopted multiple measures, including, but not limited, to establishing ne ...
   Metadata:	 {'page_label': '56', 'file_name': 'lyft_2021.pdf'}
   Score:	 0.808
   -----
   Text:	 storing unrented and returned vehicles. These impacts to the demand for and operations of the different rental programs have and may continue to adversely affectour business, financial condi tion and results of operation.• The COVID-19 pandemic may delay or prevent us, or our current or prospective partners and suppliers, from being able to test, develop or deploy autonomousvehicle-related  technology,  including  through  direct  impacts  of  the  COVID-19  virus  on  employee  and  contractor  health;  reduced  consumer  demand  forautonomous vehicle  travel resulting from an overall reduced demand for travel; shelter-in-place orders by local, state or federal governments negatively impactingoperations,  including our ability to test autonomous vehicle-related technology; impacts to the supply chains of our current or prospective partners and suppliers;or  economic  impacts  limiting  our  or  our  current  or  prospective  partners’  or  suppliers’  ability  to  expend  resources  o ...
   Metadata:	 {'page_label': '18', 'file_name': 'lyft_2021.pdf'}
   Score:	 0.805


Auto-Retrieval from a Vector Database

This guide shows how to perform **auto-retrieval** in LlamaIndex.
Many popular vector dbs support a set of metadata filters in addition
to a query string for semantic search. Given a natural language query,
we first use the LLM to infer a set of metadata filters as well as the
right query string to pass to the vector db (either can also be
blank). This overall query bundle is then executed against the vector
db.
This allows for more dynamic, expressive forms of retrieval beyond
top-k semantic search. The relevant context for a given query may only
require filtering on a metadata tag, or require a joint combination of
filtering + semantic search within the filtered set, or just raw
semantic search.
We demonstrate an example with Elasticsearch, but auto-retrieval is
also implemented with many other vector dbs (e.g. Pinecone, Weaviate,
and more).
Setup
We first define imports.
   import logging
   import sys
   logging.basicConfig(stream=sys.stdout, level=logging.INFO)
   logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))
   # set up OpenAI
   import os
   import getpass
   os.environ["OPENAI_API_KEY"] = getpass.getpass("OpenAI API Key:")
   import openai
   openai.api_key = os.environ["OPENAI_API_KEY"]
Defining Some Sample Data
We insert some sample nodes containing text chunks into the vector
database. Note that each "TextNode" not only contains the text, but
also metadata e.g. "category" and "country". These metadata fields
will get converted/stored as such in the underlying vector db.
   from llama_index import VectorStoreIndex, StorageContext
   from llama_index.vector_stores import ElasticsearchStore
   from llama_index.schema import TextNode
   nodes = [
       TextNode(
           text="A bunch of scientists bring back dinosaurs and mayhem breaks loose",
           metadata={"year": 1993, "rating": 7.7, "genre": "science fiction"},
       ),
       TextNode(
           text="Leo DiCaprio gets lost in a dream within a dream within a dream within a ...",
           metadata={"year": 2010, "director": "Christopher Nolan", "rating": 8.2},
       ),
       TextNode(
           text="A psychologist / detective gets lost in a series of dreams within dreams within dreams and Inception reused the idea",
           metadata={"year": 2006, "director": "Satoshi Kon", "rating": 8.6},
       ),
       TextNode(
           text="A bunch of normal-sized women are supremely wholesome and some men pine after them",
           metadata={"year": 2019, "director": "Greta Gerwig", "rating": 8.3},
       ),
       TextNode(
           text="Toys come alive and have a blast doing so",
           metadata={"year": 1995, "genre": "animated"},
       ),
   ]
Build Vector Index with Elasticsearch Vector Store
Here we load the data into the vector store. As mentioned above, both
the text and metadata for each node will get converted into
corresponding representation in Elasticsearch. We can now run semantic
queries and also metadata filtering on this data from Elasticsearch.
   vector_store = ElasticsearchStore(
       index_name="auto_retriever_movies", es_url="http://localhost:9200"
   )
   storage_context = StorageContext.from_defaults(vector_store=vector_store)
   index = VectorStoreIndex(nodes, storage_context=storage_context)
Define "VectorIndexAutoRetriever"
We define our core "VectorIndexAutoRetriever" module. The module takes
in "VectorStoreInfo", which contains a structured description of the
vector store collection and the metadata filters it supports. This
information will then be used in the auto-retrieval prompt where the

Auto-Retrieval from a Vector Database

LLM infers metadata filters.
   from llama_index.indices.vector_store.retrievers import VectorIndexAutoRetriever
   from llama_index.vector_stores.types import MetadataInfo, VectorStoreInfo
   vector_store_info = VectorStoreInfo(
       content_info="Brief summary of a movie",
       metadata_info=[
           MetadataInfo(
               name="genre",
               description="The genre of the movie",
               type="string or list[string]",
           ),
           MetadataInfo(
               name="year",
               description="The year the movie was released",
               type="integer",
           ),
           MetadataInfo(
               name="director",
               description="The name of the movie director",
               type="string",
           ),
           MetadataInfo(
               name="rating", description="A 1-10 rating for the movie", type="float"
           ),
       ],
   )
   retriever = VectorIndexAutoRetriever(index, vector_store_info=vector_store_info)
Running over some sample data
We try running over some sample data. Note how metadata filters are
inferred - this helps with more precise retrieval!
   retriever.retrieve("What are 2 movies by Christopher Nolan were made before 2020?")
   retriever.retrieve("Has Andrei Tarkovsky directed any science fiction movies")
   INFO:llama_index.indices.vector_store.retrievers.auto_retriever.auto_retriever:Using query str: science fiction
   Using query str: science fiction
   INFO:llama_index.indices.vector_store.retrievers.auto_retriever.auto_retriever:Using filters: {'director': 'Andrei Tarkovsky'}
   Using filters: {'director': 'Andrei Tarkovsky'}
   INFO:llama_index.indices.vector_store.retrievers.auto_retriever.auto_retriever:Using top_k: 2
   Using top_k: 2
   INFO:elastic_transport.transport:POST http://localhost:9200/auto_retriever_movies/_search [status:200 duration:0.042s]
   POST http://localhost:9200/auto_retriever_movies/_search [status:200 duration:0.042s]
   []


Elasticsearch

   Elasticsearch is a search database, that supports full text and
   vector searches.
Basic Example
In this basic example, we take the a Paul Graham essay, split it into
chunks, embed it using an open-source embedding model, load it into
Elasticsearch, and then query it.
   # !pip install llama-index elasticsearch --quiet
   # !pip install sentence-transformers
   # !pip install pydantic==1.10.11
   # import
   from llama_index import VectorStoreIndex, SimpleDirectoryReader, ServiceContext
   from llama_index.vector_stores import ElasticsearchStore
   from llama_index.storage.storage_context import StorageContext
   from IPython.display import Markdown, display
   # set up OpenAI
   import os
   import getpass
   os.environ["OPENAI_API_KEY"] = getpass.getpass("OpenAI API Key:")
   import openai
   openai.api_key = os.environ["OPENAI_API_KEY"]
   # define embedding function
   embed_model = "local/BAAI/bge-small-en-v1.5"
   # load documents
   documents = SimpleDirectoryReader(
       "../../../examples/paul_graham_essay/data"
   ).load_data()
   vector_store = ElasticsearchStore(
       index_name="paul_graham_essay", es_url="http://localhost:9200"
   )
   storage_context = StorageContext.from_defaults(vector_store=vector_store)
   service_context = ServiceContext.from_defaults(embed_model=embed_model)
   index = VectorStoreIndex.from_documents(
       documents, storage_context=storage_context, service_context=service_context
   )
   # Query Data
   query_engine = index.as_query_engine()
   response = query_engine.query("What did the author do growing up?")
   display(Markdown(f"<b>{response}</b>"))
The author worked on writing and programming outside of school. They
wrote short stories and tried writing programs on an IBM 1401
computer. They also built a microcomputer kit and started programming
on it, writing simple games and a word processor.


Zep Vector Store

A long-term memory store for LLM applications
This notebook demonstrates how to use the Zep Vector Store with
LlamaIndex.
About Zep
Zep makes it easy for developers to add relevant documents, chat
history memory & rich user data to their LLM app's prompts.
Note
Zep can automatically embed your documents. The LlamaIndex
implementation of the Zep Vector Store utilizes LlamaIndex's embedders
to do so.
Getting Started
**Quick Start Guide:** https://docs.getzep.com/deployment/quickstart/
**GitHub:** https://github.com/getzep/zep
   # !pip install zep-python
   import logging
   import sys
   from uuid import uuid4
   logging.basicConfig(stream=sys.stdout, level=logging.INFO)
   logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))
   import os
   import openai
   from dotenv import load_dotenv
   load_dotenv()
   # os.environ["OPENAI_API_KEY"] = "sk-..."
   openai.api_key = os.environ["OPENAI_API_KEY"]
   from llama_index import VectorStoreIndex, SimpleDirectoryReader
   from llama_index.vector_stores.zep import ZepVectorStore
   INFO:numexpr.utils:NumExpr defaulting to 8 threads.
   NumExpr defaulting to 8 threads.
   # load documents
   documents = SimpleDirectoryReader("../data/paul_graham").load_data()
Create a Zep Vector Store and Index
You can use an existing Zep Collection, or create a new one.
   from llama_index.storage.storage_context import StorageContext
   zep_api_url = "http://localhost:8000"
   collection_name = f"graham{uuid4().hex}"
   vector_store = ZepVectorStore(
       api_url=zep_api_url, collection_name=collection_name, embedding_dimensions=1536
   )
   storage_context = StorageContext.from_defaults(vector_store=vector_store)
   index = VectorStoreIndex.from_documents(documents, storage_context=storage_context)
   INFO:httpx:HTTP Request: GET http://localhost:8000/healthz "HTTP/1.1 200 OK"
   HTTP Request: GET http://localhost:8000/healthz "HTTP/1.1 200 OK"
   INFO:httpx:HTTP Request: GET http://localhost:8000/api/v1/collection/grahamfbf0c456a2ad46c2887a707ccc7bb5df "HTTP/1.1 404 Not Found"
   HTTP Request: GET http://localhost:8000/api/v1/collection/grahamfbf0c456a2ad46c2887a707ccc7bb5df "HTTP/1.1 404 Not Found"
   INFO:llama_index.vector_stores.zep:Collection grahamfbf0c456a2ad46c2887a707ccc7bb5df does not exist, will try creating one with dimensions=1536
   Collection grahamfbf0c456a2ad46c2887a707ccc7bb5df does not exist, will try creating one with dimensions=1536
   INFO:httpx:HTTP Request: POST http://localhost:8000/api/v1/collection/grahamfbf0c456a2ad46c2887a707ccc7bb5df "HTTP/1.1 200 OK"
   HTTP Request: POST http://localhost:8000/api/v1/collection/grahamfbf0c456a2ad46c2887a707ccc7bb5df "HTTP/1.1 200 OK"
   INFO:httpx:HTTP Request: GET http://localhost:8000/api/v1/collection/grahamfbf0c456a2ad46c2887a707ccc7bb5df "HTTP/1.1 200 OK"

Zep Vector Store

   HTTP Request: GET http://localhost:8000/api/v1/collection/grahamfbf0c456a2ad46c2887a707ccc7bb5df "HTTP/1.1 200 OK"
   INFO:httpx:HTTP Request: POST http://localhost:8000/api/v1/collection/grahamfbf0c456a2ad46c2887a707ccc7bb5df/document "HTTP/1.1 200 OK"
   HTTP Request: POST http://localhost:8000/api/v1/collection/grahamfbf0c456a2ad46c2887a707ccc7bb5df/document "HTTP/1.1 200 OK"
   query_engine = index.as_query_engine()
   response = query_engine.query("What did the author do growing up?")
   print(str(response))
   INFO:httpx:HTTP Request: POST http://localhost:8000/api/v1/collection/grahamfbf0c456a2ad46c2887a707ccc7bb5df/search?limit=2 "HTTP/1.1 200 OK"
   HTTP Request: POST http://localhost:8000/api/v1/collection/grahamfbf0c456a2ad46c2887a707ccc7bb5df/search?limit=2 "HTTP/1.1 200 OK"
   The author worked on writing and programming outside of school before college. They wrote short stories and tried writing programs on an IBM 1401 computer using an early version of Fortran. They later got a microcomputer and started programming more extensively, writing simple games, a program to predict rocket heights, and a word processor. They initially planned to study philosophy in college but switched to AI. They also started publishing essays online and realized the potential of the web as a medium for publishing.
Querying with Metadata filters
   from llama_index.schema import TextNode
   nodes = [
       TextNode(
           text="The Shawshank Redemption",
           metadata={
               "author": "Stephen King",
               "theme": "Friendship",
           },
       ),
       TextNode(
           text="The Godfather",
           metadata={
               "director": "Francis Ford Coppola",
               "theme": "Mafia",
           },
       ),
       TextNode(
           text="Inception",
           metadata={
               "director": "Christopher Nolan",
           },
       ),
   ]
   collection_name = f"movies{uuid4().hex}"
   vector_store = ZepVectorStore(
       api_url=zep_api_url, collection_name=collection_name, embedding_dimensions=1536
   )
   storage_context = StorageContext.from_defaults(vector_store=vector_store)
   index = VectorStoreIndex(nodes, storage_context=storage_context)
   INFO:httpx:HTTP Request: GET http://localhost:8000/healthz "HTTP/1.1 200 OK"
   HTTP Request: GET http://localhost:8000/healthz "HTTP/1.1 200 OK"
   INFO:httpx:HTTP Request: GET http://localhost:8000/api/v1/collection/movies40ffd4f8a68c4822ae1680bb752c07e1 "HTTP/1.1 404 Not Found"
   HTTP Request: GET http://localhost:8000/api/v1/collection/movies40ffd4f8a68c4822ae1680bb752c07e1 "HTTP/1.1 404 Not Found"
   INFO:llama_index.vector_stores.zep:Collection movies40ffd4f8a68c4822ae1680bb752c07e1 does not exist, will try creating one with dimensions=1536
   Collection movies40ffd4f8a68c4822ae1680bb752c07e1 does not exist, will try creating one with dimensions=1536

Zep Vector Store

   INFO:httpx:HTTP Request: POST http://localhost:8000/api/v1/collection/movies40ffd4f8a68c4822ae1680bb752c07e1 "HTTP/1.1 200 OK"
   HTTP Request: POST http://localhost:8000/api/v1/collection/movies40ffd4f8a68c4822ae1680bb752c07e1 "HTTP/1.1 200 OK"
   INFO:httpx:HTTP Request: GET http://localhost:8000/api/v1/collection/movies40ffd4f8a68c4822ae1680bb752c07e1 "HTTP/1.1 200 OK"
   HTTP Request: GET http://localhost:8000/api/v1/collection/movies40ffd4f8a68c4822ae1680bb752c07e1 "HTTP/1.1 200 OK"
   INFO:httpx:HTTP Request: POST http://localhost:8000/api/v1/collection/movies40ffd4f8a68c4822ae1680bb752c07e1/document "HTTP/1.1 200 OK"
   HTTP Request: POST http://localhost:8000/api/v1/collection/movies40ffd4f8a68c4822ae1680bb752c07e1/document "HTTP/1.1 200 OK"
   from llama_index.vector_stores.types import ExactMatchFilter, MetadataFilters
   filters = MetadataFilters(filters=[ExactMatchFilter(key="theme", value="Mafia")])
   retriever = index.as_retriever(filters=filters)
   result = retriever.retrieve("What is inception about?")
   for r in result:
       print("\n", r.node)
       print("Score:", r.score)
   INFO:httpx:HTTP Request: POST http://localhost:8000/api/v1/collection/movies40ffd4f8a68c4822ae1680bb752c07e1/search?limit=2 "HTTP/1.1 200 OK"
   HTTP Request: POST http://localhost:8000/api/v1/collection/movies40ffd4f8a68c4822ae1680bb752c07e1/search?limit=2 "HTTP/1.1 200 OK"
    Node ID: 2b5ad50a-8ec0-40fa-b401-6e6b7ac3d304
   Text: The Godfather
   Score: 0.8841066656525941


Typesense Vector Store

Load documents, build the VectorStoreIndex
   # import logging
   # import sys
   # logging.basicConfig(stream=sys.stdout, level=logging.INFO)
   # logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))
   from llama_index import VectorStoreIndex, SimpleDirectoryReader, StorageContext
   from IPython.display import Markdown, display
   # load documents
   documents = SimpleDirectoryReader(
       "../../../examples/paul_graham_essay/data"
   ).load_data()
   from llama_index.vector_stores.typesense import TypesenseVectorStore
   from typesense import Client
   typesense_client = Client(
       {
           "api_key": "xyz",
           "nodes": [{"host": "localhost", "port": "8108", "protocol": "http"}],
           "connection_timeout_seconds": 2,
       }
   )
   typesense_vector_store = TypesenseVectorStore(typesense_client)
   storage_context = StorageContext.from_defaults(vector_store=typesense_vector_store)
   index = VectorStoreIndex.from_documents(documents, storage_context=storage_context)
Query Index
   from llama_index.indices.query.schema import QueryBundle
   from llama_index.embeddings import OpenAIEmbedding
   # By default, typesense vector store uses vector search. You need to provide the embedding yourself.
   query_str = "What did the author do growing up?"
   embed_model = OpenAIEmbedding()
   # If your service context has an embed_model you can also do:
   # embed_model = index.service_context.embed_model
   query_embedding = embed_model.get_agg_embedding_from_queries(query_str)
   query_bundle = QueryBundle(query_str, embedding=query_embedding)
   response = index.as_query_engine().query(query_bundle)
   display(Markdown(f"{response}"))
The author grew up skipping a step in the evolution of computers,
learning Italian, walking through Florence, painting people, working
with technology companies, seeking signature styles at RISD, living in
a rent-stabilized apartment, launching software, editing code
(including Lisp expressions), writing essays, publishing them online,
and receiving feedback from angry readers. He also experienced the
exponential growth of commodity processors in the 1990s, which rolled
up high-end, special-purpose hardware and software companies. He also
learned how to make a little Italian go a long way by stringing
together abstract concepts with a few simple verbs. He also
experienced the tight coupling of money and coolness in the art world,
and the fact that anything expensive comes to be seen as cool, and
anything seen as cool will soon become equally expensive. He also
experienced the challenge of launching software, as he had to recruit
an initial set of users and make sure they had decent-looking stores
before launching publicly. He also experienced the first instance of
what is now a familiar experience, when he read the comments and found
they were full of angry people. He also experienced the difference
between putting something online and publishing it online. Finally, he
wrote essays about topics he had stacked up, and wrote a more detailed
version for others to read.
   from llama_index.vector_stores.types import VectorStoreQueryMode
   # You can also use text search
   query_bundle = QueryBundle(query_str=query_str)
   response = index.as_query_engine(
       vector_store_query_mode=VectorStoreQueryMode.TEXT_SEARCH
   ).query(query_bundle)
   display(Markdown(f"{response}"))
The author grew up during the Internet Bubble and was running a
startup. They had to hire more people than they wanted to in order to
seem more professional and were at the mercy of their investors until
Yahoo bought them. They learned a lot about retail and startups, and
had to do a lot of things that they weren't necessarily good at in

Typesense Vector Store

order to make their business successful.


Bagel Vector Store

   import logging
   import sys
   logging.basicConfig(stream=sys.stdout, level=logging.INFO)
   logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))
   # set up OpenAI
   import os
   import getpass
   os.environ["OPENAI_API_KEY"] = getpass.getpass("OpenAI API Key:")
   import openai
   openai.api_key = os.environ["OPENAI_API_KEY"]
   import bagel
   from bagel import Settings
   server_settings = Settings(bagel_api_impl="rest", bagel_server_host="api.bageldb.ai")
   client = bagel.Client(server_settings)
   collection = client.get_or_create_cluster("testing_embeddings")
   from llama_index import VectorStoreIndex, StorageContext
   from llama_index.vector_stores import BagelVectorStore
   from llama_index.schema import TextNode
   nodes = [
       TextNode(
           text="Michael Jordan is a retired professional basketball player, widely regarded as one of the greatest basketball players of all time.",
           metadata={
               "category": "Sports",
               "country": "United States",
           },
       ),
       TextNode(
           text="Angelina Jolie is an American actress, filmmaker, and humanitarian. She has received numerous awards for her acting and is known for her philanthropic work.",
           metadata={
               "category": "Entertainment",
               "country": "United States",
           },
       ),
       TextNode(
           text="Elon Musk is a business magnate, industrial designer, and engineer. He is the founder, CEO, and lead designer of SpaceX, Tesla, Inc., Neuralink, and The Boring Company.",
           metadata={
               "category": "Business",
               "country": "United States",
           },
       ),
       TextNode(
           text="Rihanna is a Barbadian singer, actress, and businesswoman. She has achieved significant success in the music industry and is known for her versatile musical style.",
           metadata={
               "category": "Music",
               "country": "Barbados",
           },
       ),
       TextNode(
           text="Cristiano Ronaldo is a Portuguese professional footballer who is considered one of the greatest football players of all time. He has won numerous awards and set multiple records during his career.",
           metadata={
               "category": "Sports",
               "country": "Portugal",
           },
       ),
   ]
   vector_store = BagelVectorStore(collection=collection)
   storage_context = StorageContext.from_defaults(vector_store=vector_store)
   index = VectorStoreIndex(nodes, storage_context=storage_context)
   from llama_index.indices.vector_store.retrievers import VectorIndexAutoRetriever
   from llama_index.vector_stores.types import MetadataInfo, VectorStoreInfo
   vector_store_info = VectorStoreInfo(
       content_info="brief biography of celebrities",
       metadata_info=[
           MetadataInfo(
               name="category",
               type="str",
               description="Category of the celebrity, one of [Sports, Entertainment, Business, Music]",
           ),
           MetadataInfo(
               name="country",
               type="str",
               description="Country of the celebrity, one of [United States, Barbados, Portugal]",
           ),
       ],
   )
   retriever = VectorIndexAutoRetriever(index, vector_store_info=vector_store_info)
   retriever.retrieve("Tell me about two celebrities from United States")


Cassandra Vector Store

Apache Cassandra® is a NoSQL, row-oriented, highly scalable and highly
available database. Newest Cassandra releases natively support Vector
Similarity Search.
**This notebook shows the basic usage of Cassandra as a Vector Store
in LlamaIndex.**
To run this notebook you need either a running Cassandra cluster
equipped with Vector Search capabilities (in pre-release at the time
of writing) or a DataStax Astra DB instance running in the cloud (you
can get one for free at datastax.com). *This notebook covers both
choices.* Check cassio.org for more information, quickstarts and
tutorials.
Setup
   !pip install "cassio>=0.1.0"
   import os
   from llama_index import (
       VectorStoreIndex,
       SimpleDirectoryReader,
       Document,
       StorageContext,
   )
   from llama_index.vector_stores import CassandraVectorStore
Please provide database connection parameters and secrets
First you need a database connection (a "cassandra.cluster.Session"
object).
Make sure you have either a vector-capable running Cassandra cluster
or an Astra DB instance in the cloud.
   import os
   import getpass
   database_mode = (input("\n(C)assandra or (A)stra DB? ")).upper()
   keyspace_name = input("\nKeyspace name? ")
   if database_mode == "A":
       ASTRA_DB_APPLICATION_TOKEN = getpass.getpass('\nAstra DB Token ("AstraCS:...") ')
       #
       ASTRA_DB_SECURE_BUNDLE_PATH = input("Full path to your Secure Connect Bundle? ")
   elif database_mode == "C":
       CASSANDRA_CONTACT_POINTS = input(
           "Contact points? (comma-separated, empty for localhost) "
       ).strip()
   from cassandra.cluster import Cluster
   from cassandra.auth import PlainTextAuthProvider
   if database_mode == "C":
       if CASSANDRA_CONTACT_POINTS:
           cluster = Cluster(
               [cp.strip() for cp in CASSANDRA_CONTACT_POINTS.split(",") if cp.strip()]
           )
       else:
           cluster = Cluster()
       session = cluster.connect()
   elif database_mode == "A":
       ASTRA_DB_CLIENT_ID = "token"
       cluster = Cluster(
           cloud={
               "secure_connect_bundle": ASTRA_DB_SECURE_BUNDLE_PATH,
           },
           auth_provider=PlainTextAuthProvider(
               ASTRA_DB_CLIENT_ID,
               ASTRA_DB_APPLICATION_TOKEN,
           ),
       )
       session = cluster.connect()
   else:
       raise NotImplementedError
Please provide OpenAI access key
In order use embeddings by OpenAI you need to supply an OpenAI API
Key:
   import openai
   OPENAI_API_KEY = getpass.getpass("OpenAI API Key:")
   openai.api_key = OPENAI_API_KEY
Creating and populating the Vector Store
You will now load some essays by Paul Graham from a local file and
store them into the Cassandra Vector Store.
   # load documents
   documents = SimpleDirectoryReader("../data/paul_graham").load_data()
   print(f"Total documents: {len(documents)}")
   print(f"First document, id: {documents[0].doc_id}")
   print(f"First document, hash: {documents[0].hash}")
   print(
       f"First document, text ({len(documents[0].text)} characters):\n{'='*20}\n{documents[0].text[:360]} ..."
   )
   Total documents: 1
   First document, id: 5b7489b6-0cca-4088-8f30-6de32d540fdf
   First document, hash: 4c702b4df575421e1d1af4b1fd50511b226e0c9863dbfffeccb8b689b8448f35

Cassandra Vector Store

   First document, text (75019 characters):
   ====================
   What I Worked On
   February 2021
   Before college the two main things I worked on, outside of school, were writing and programming. I didn't write essays. I wrote what beginning writers were supposed to write then, and probably still are: short stories. My stories were awful. They had hardly any plot, just characters with strong feelings, which I imagined  ...
Initialize the Cassandra Vector Store
Creation of the vector store entails creation of the underlying
database table if it does not exist yet:
   cassandra_store = CassandraVectorStore(
       session=session,
       keyspace=keyspace_name,
       table="cassandra_vector_table_1",
       embedding_dimension=1536,
   )
Now wrap this store into an "index" LlamaIndex abstraction for later
querying:
   storage_context = StorageContext.from_defaults(vector_store=cassandra_store)
   index = VectorStoreIndex.from_documents(documents, storage_context=storage_context)
Note that the above "from_documents" call does several things at once:
it splits the input documents into chunks of manageable size
("nodes"), computes embedding vectors for each node, and stores them
all in the Cassandra Vector Store.
Querying the store
Basic querying
   query_engine = index.as_query_engine()
   response = query_engine.query("Why did the author choose to work on AI?")
   print(response.response)
   The author chose to work on AI because of his fascination with the novel The Moon is a Harsh Mistress, which featured an intelligent computer called Mike, and a PBS documentary that showed Terry Winograd using SHRDLU. He was also drawn to the idea that AI could be used to explore the ultimate truths that other fields could not.
MMR-based queries
The MMR (maximal marginal relevance) method is designed to fetch text
chunks from the store that are at the same time relevant to the query
but as different as possible from each other, with the goal of
providing a broader context to the building of the final answer:
   query_engine = index.as_query_engine(vector_store_query_mode="mmr")
   response = query_engine.query("Why did the author choose to work on AI?")
   print(response.response)
   The author chose to work on AI because he was impressed and envious of his friend who had built a computer kit and was able to type programs into it. He was also inspired by a novel by Heinlein called The Moon is a Harsh Mistress, which featured an intelligent computer called Mike, and a PBS documentary that showed Terry Winograd using SHRDLU. He was also disappointed with philosophy courses in college, which he found to be boring, and he wanted to work on something that seemed more powerful.
Connecting to an existing store
Since this store is backed by Cassandra, it is persistent by
definition. So, if you want to connect to a store that was created and
populated previously, here is how:
   new_store_instance = CassandraVectorStore(
       session=session,
       keyspace=keyspace_name,
       table="cassandra_vector_table_1",
       embedding_dimension=1536,
   )
   # Create index (from preexisting stored vectors)
   new_index_instance = VectorStoreIndex.from_vector_store(vector_store=new_store_instance)
   # now you can do querying, etc:
   query_engine = index.as_query_engine(similarity_top_k=5)
   response = query_engine.query("What did the author study prior to working on AI?")
   print(response.response)
   The author studied philosophy and painting, worked on spam filters, and wrote essays prior to working on AI.
Removing documents from the index
First get an explicit list of pieces of a document, or "nodes", from a
"Retriever" spawned from the index:
   retriever = new_index_instance.as_retriever(
       vector_store_query_mode="mmr",

Cassandra Vector Store

       similarity_top_k=3,
       vector_store_kwargs={"mmr_prefetch_factor": 4},
   )
   nodes_with_scores = retriever.retrieve(
       "What did the author study prior to working on AI?"
   )
   print(f"Found {len(nodes_with_scores)} nodes.")
   for idx, node_with_score in enumerate(nodes_with_scores):
       print(f"    [{idx}] score = {node_with_score.score}")
       print(f"        id    = {node_with_score.node.node_id}")
       print(f"        text  = {node_with_score.node.text[:90]} ...")
   Found 3 nodes.
       [0] score = 0.42589144520149874
           id    = 05f53f06-9905-461a-bc6d-fa4817e5a776
           text  = What I Worked On
   February 2021
   Before college the two main things I worked on, outside o ...
       [1] score = -0.0012061281453193962
           id    = 2f9f843e-6495-4646-a03d-4b844ff7c1ab
           text  = been explored. But all I wanted was to get out of grad school, and my rapidly written diss ...
       [2] score = 0.025454533089838027
           id    = 28ad32da-25f9-4aaa-8487-88390ec13348
           text  = showed Terry Winograd using SHRDLU. I haven't tried rereading The Moon is a Harsh Mistress ...
But wait! When using the vector store, you should consider the
**document** as the sensible unit to delete, and not any individual
node belonging to it. Well, in this case, you just inserted a single
text file, so all nodes will have the same "ref_doc_id":
   print("Nodes' ref_doc_id:")
   print("\n".join([nws.node.ref_doc_id for nws in nodes_with_scores]))
   Nodes' ref_doc_id:
   5b7489b6-0cca-4088-8f30-6de32d540fdf
   5b7489b6-0cca-4088-8f30-6de32d540fdf
   5b7489b6-0cca-4088-8f30-6de32d540fdf
Now let's say you need to remove the text file you uploaded:
   new_store_instance.delete(nodes_with_scores[0].node.ref_doc_id)
Repeat the very same query and check the results now. You should see
*no results* being found:
   nodes_with_scores = retriever.retrieve(
       "What did the author study prior to working on AI?"
   )
   print(f"Found {len(nodes_with_scores)} nodes.")
   Found 0 nodes.
Metadata filtering
The Cassandra vector store support metadata filtering in the form of
exact-match "key=value" pairs at query time. The following cells,
which work on a brand new Cassandra table, demonstrate this feature.
In this demo, for the sake of brevity, a single source document is
loaded (the "../data/paul_graham/paul_graham_essay.txt" text file).
Nevertheless, you will attach some custom metadata to the document to
illustrate how you can can restrict queries with conditions on the
metadata attached to the documents.
   md_storage_context = StorageContext.from_defaults(
       vector_store=CassandraVectorStore(
           session=session,
           keyspace=keyspace_name,
           table="cassandra_vector_table_2_md",
           embedding_dimension=1536,
       )
   )
   def my_file_metadata(file_name: str):
       """Depending on the input file name, associate a different metadata."""

Cassandra Vector Store

       if "essay" in file_name:
           source_type = "essay"
       elif "dinosaur" in file_name:
           # this (unfortunately) will not happen in this demo
           source_type = "dinos"
       else:
           source_type = "other"
       return {"source_type": source_type}
   # Load documents and build index
   md_documents = SimpleDirectoryReader(
       "../data/paul_graham", file_metadata=my_file_metadata
   ).load_data()
   md_index = VectorStoreIndex.from_documents(
       md_documents, storage_context=md_storage_context
   )
That's it: you can now add filtering to your query engine:
   from llama_index.vector_stores.types import ExactMatchFilter, MetadataFilters
   md_query_engine = md_index.as_query_engine(
       filters=MetadataFilters(
           filters=[ExactMatchFilter(key="source_type", value="essay")]
       )
   )
   md_response = md_query_engine.query("How long it took the author to write his thesis?")
   print(md_response.response)
   It took the author five weeks to write his thesis.
To test that the filtering is at play, try to change it to use only
""dinos"" documents... there will be no answer this time :)


Pinecone Vector Store - Hybrid Search

Creating a Pinecone Index
   import logging
   import sys
   logging.basicConfig(stream=sys.stdout, level=logging.INFO)
   logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))
   import pinecone
   api_key = ""
   pinecone.init(api_key=api_key, environment="us-west1-gcp")
   pinecone.describe_index("quickstart")
   # dimensions are for text-embedding-ada-002
   # NOTE: needs dotproduct for hybrid search
   pinecone.create_index("quickstart", dimension=1536, metric="dotproduct", pod_type="p1")
   pinecone_index = pinecone.Index("quickstart")
Load documents, build the PineconeVectorStore
   from llama_index import VectorStoreIndex, SimpleDirectoryReader
   from llama_index.vector_stores import PineconeVectorStore
   from IPython.display import Markdown, display
   # load documents
   documents = SimpleDirectoryReader("../paul_graham_essay/data").load_data()
   # set add_sparse_vector=True to compute sparse vectors during upsert
   from llama_index.storage.storage_context import StorageContext
   vector_store = PineconeVectorStore(
       pinecone_index=pinecone_index,
       add_sparse_vector=True,
   )
   storage_context = StorageContext.from_defaults(vector_store=vector_store)
   index = VectorStoreIndex.from_documents(documents, storage_context=storage_context)
Query Index
   # set Logging to DEBUG for more detailed outputs
   query_engine = index.as_query_engine(vector_store_query_mode="hybrid")
   response = query_engine.query("What did the author do growing up?")
   display(Markdown(f"<b>{response}</b>"))


Chroma

   Chroma is a AI-native open-source vector database focused on
   developer productivity and happiness. Chroma is licensed under
   Apache 2.0.
* Website
* Documentation
* Twitter
* Discord
Chroma is fully-typed, fully-tested and fully-documented.
Install Chroma with:
   pip install chromadb
Chroma runs in various modes. See below for examples of each
integrated with LangChain.
* "in-memory" - in a python script or jupyter notebook
* "in-memory with persistance" - in a script or notebook and save/load
  to disk
* "in a docker container" - as a server running your local machine or
  in the cloud
Like any other database, you can:
* ".add"
* ".get"
* ".update"
* ".upsert"
* ".delete"
* ".peek"
* and ".query" runs the similarity search.
View full docs at docs.
Basic Example
In this basic example, we take the a Paul Graham essay, split it into
chunks, embed it using an open-source embedding model, load it into
Chroma, and then query it.
Creating a Chroma Index
   # !pip install llama-index chromadb --quiet
   # !pip install chromadb
   # !pip install sentence-transformers
   # !pip install pydantic==1.10.11
   # import
   from llama_index import VectorStoreIndex, SimpleDirectoryReader, ServiceContext
   from llama_index.vector_stores import ChromaVectorStore
   from llama_index.storage.storage_context import StorageContext
   from llama_index.embeddings import HuggingFaceEmbedding
   from IPython.display import Markdown, display
   import chromadb
   # set up OpenAI
   import os
   import getpass
   os.environ["OPENAI_API_KEY"] = getpass.getpass("OpenAI API Key:")
   import openai
   openai.api_key = os.environ["OPENAI_API_KEY"]
   # create client and a new collection
   chroma_client = chromadb.EphemeralClient()
   chroma_collection = chroma_client.create_collection("quickstart")
   # define embedding function
   embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-base-en-v1.5")
   # load documents
   documents = SimpleDirectoryReader(
       "../../../examples/paul_graham_essay/data"
   ).load_data()
   # set up ChromaVectorStore and load in data
   vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
   storage_context = StorageContext.from_defaults(vector_store=vector_store)
   service_context = ServiceContext.from_defaults(embed_model=embed_model)
   index = VectorStoreIndex.from_documents(
       documents, storage_context=storage_context, service_context=service_context
   )
   # Query Data
   query_engine = index.as_query_engine()
   response = query_engine.query("What did the author do growing up?")
   display(Markdown(f"<b>{response}</b>"))
   /Users/loganmarkewich/llama_index/llama-index/lib/python3.9/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
     from .autonotebook import tqdm as notebook_tqdm
   /Users/loganmarkewich/llama_index/llama-index/lib/python3.9/site-packages/bitsandbytes/cextension.py:34: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers, 8-bit multiplication, and GPU quantization are unavailable.
     warn("The installed version of bitsandbytes was compiled without GPU support. "

Chroma

'NoneType' object has no attribute 'cadam32bit_grad_fp32'
The author worked on writing and programming growing up. They wrote
short stories and tried writing programs on an IBM 1401 computer.
Later, they got a microcomputer and started programming more
extensively.
Basic Example (including saving to disk)
Extending the previous example, if you want to save to disk, simply
initialize the Chroma client and pass the directory where you want the
data to be saved to.
"Caution": Chroma makes a best-effort to automatically save data to
disk, however multiple in-memory clients can stomp each other's work.
As a best practice, only have one client per path running at any given
time.
   # save to disk
   db = chromadb.PersistentClient(path="./chroma_db")
   chroma_collection = db.get_or_create_collection("quickstart")
   vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
   storage_context = StorageContext.from_defaults(vector_store=vector_store)
   service_context = ServiceContext.from_defaults(embed_model=embed_model)
   index = VectorStoreIndex.from_documents(
       documents, storage_context=storage_context, service_context=service_context
   )
   # load from disk
   db2 = chromadb.PersistentClient(path="./chroma_db")
   chroma_collection = db2.get_or_create_collection("quickstart")
   vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
   index = VectorStoreIndex.from_vector_store(
       vector_store,
       service_context=service_context,
   )
   # Query Data from the persisted index
   query_engine = index.as_query_engine()
   response = query_engine.query("What did the author do growing up?")
   display(Markdown(f"{response}"))
The author worked on writing and programming growing up. They wrote
short stories and tried writing programs on an IBM 1401 computer.
Later, they got a microcomputer and started programming games and a
word processor.
Basic Example (using the Docker Container)
You can also run the Chroma Server in a Docker container separately,
create a Client to connect to it, and then pass that to LlamaIndex.
Here is how to clone, build, and run the Docker Image:
   git clone git@github.com:chroma-core/chroma.git
   docker-compose up -d --build
   # create the chroma client and add our data
   import chromadb
   remote_db = chromadb.HttpClient()
   chroma_collection = remote_db.get_or_create_collection("quickstart")
   vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
   storage_context = StorageContext.from_defaults(vector_store=vector_store)
   service_context = ServiceContext.from_defaults(embed_model=embed_model)
   index = VectorStoreIndex.from_documents(
       documents, storage_context=storage_context, service_context=service_context
   )
   # Query Data from the Chroma Docker index
   query_engine = index.as_query_engine()
   response = query_engine.query("What did the author do growing up?")
   display(Markdown(f"{response}"))
Update and Delete
While building toward a real application, you want to go beyond adding
data, and also update and delete data.
Chroma has users provide "ids" to simplify the bookkeeping here. "ids"
can be the name of the file, or a combined has like
"filename_paragraphNumber", etc.
Here is a basic example showing how to do various operations:
   doc_to_update = chroma_collection.get(limit=1)
   doc_to_update["metadatas"][0] = {
       **doc_to_update["metadatas"][0],
       **{"author": "Paul Graham"},
   }
   chroma_collection.update(
       ids=[doc_to_update["ids"][0]], metadatas=[doc_to_update["metadatas"][0]]

Chroma

   )
   updated_doc = chroma_collection.get(limit=1)
   print(updated_doc["metadatas"][0])
   # delete the last document
   print("count before", chroma_collection.count())
   chroma_collection.delete(ids=[doc_to_update["ids"][0]])
   print("count after", chroma_collection.count())
   {'_node_content': '{"id_": "be08c8bc-f43e-4a71-ba64-e525921a8319", "embedding": null, "metadata": {}, "excluded_embed_metadata_keys": [], "excluded_llm_metadata_keys": [], "relationships": {"1": {"node_id": "2cbecdbb-0840-48b2-8151-00119da0995b", "node_type": null, "metadata": {}, "hash": "4c702b4df575421e1d1af4b1fd50511b226e0c9863dbfffeccb8b689b8448f35"}, "3": {"node_id": "6a75604a-fa76-4193-8f52-c72a7b18b154", "node_type": null, "metadata": {}, "hash": "d6c408ee1fbca650fb669214e6f32ffe363b658201d31c204e85a72edb71772f"}}, "hash": "b4d0b960aa09e693f9dc0d50ef46a3d0bf5a8fb3ac9f3e4bcf438e326d17e0d8", "text": "", "start_char_idx": 0, "end_char_idx": 4050, "text_template": "{metadata_str}\\n\\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\\n"}', 'author': 'Paul Graham', 'doc_id': '2cbecdbb-0840-48b2-8151-00119da0995b', 'document_id': '2cbecdbb-0840-48b2-8151-00119da0995b', 'ref_doc_id': '2cbecdbb-0840-48b2-8151-00119da0995b'}
   count before 20
   count after 19


Pinecone Vector Store - Metadata Filter

   import logging
   import sys
   import os
   logging.basicConfig(stream=sys.stdout, level=logging.INFO)
   logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))
Build a Pinecone Index and connect to it
   import pinecone
   api_key = os.environ["PINECONE_API_KEY"]
   pinecone.init(api_key=api_key, environment="eu-west1-gcp")
   # dimensions are for text-embedding-ada-002
   pinecone.create_index(
       "quickstart-index", dimension=1536, metric="euclidean", pod_type="p1"
   )
   pinecone_index = pinecone.Index("quickstart-index")
Build the PineconeVectorStore and VectorStoreIndex
   from llama_index import VectorStoreIndex, StorageContext
   from llama_index.vector_stores import PineconeVectorStore
   from llama_index.schema import TextNode
   nodes = [
       TextNode(
           text="The Shawshank Redemption",
           metadata={
               "author": "Stephen King",
               "theme": "Friendship",
           },
       ),
       TextNode(
           text="The Godfather",
           metadata={
               "director": "Francis Ford Coppola",
               "theme": "Mafia",
           },
       ),
       TextNode(
           text="Inception",
           metadata={
               "director": "Christopher Nolan",
           },
       ),
   ]
   vector_store = PineconeVectorStore(
       pinecone_index=pinecone_index, namespace="test_05_14"
   )
   storage_context = StorageContext.from_defaults(vector_store=vector_store)
   index = VectorStoreIndex(nodes, storage_context=storage_context)
Define metadata filters
   from llama_index.vector_stores.types import ExactMatchFilter, MetadataFilters
   filters = MetadataFilters(filters=[ExactMatchFilter(key="theme", value="Mafia")])
Retrieve from vector store with filters
   retriever = index.as_retriever(filters=filters)
   retriever.retrieve("What is inception about?")
Use keyword arguments specific to pinecone
   retriever = index.as_retriever(vector_store_kwargs={"filter": {"theme": "Mafia"}})
   retriever.retrieve("What is inception about?")