Node Postprocessor

Node PostProcessor module.
pydantic model llama_index.indices.postprocessor.AutoPrevNextNodePostprocessor
   Previous/Next Node post-processor.
   Allows users to fetch additional nodes from the document store,
   based on the prev/next relationships of the nodes.
   NOTE: difference with PrevNextPostprocessor is that this infers
   forward/backwards direction.
   NOTE: this is a beta feature.
   Parameters:
      * **docstore** (*BaseDocumentStore*) -- The document store.
      * **llm_predictor** (*LLMPredictor*) -- The LLM predictor.
      * **num_nodes** (*int*) -- The number of nodes to return
        (default: 1)
      * **infer_prev_next_tmpl** (*str*) -- The template to use for
        inference. Required fields are {context_str} and {query_str}.
      {
         "title": "AutoPrevNextNodePostprocessor",
         "description": "Previous/Next Node post-processor.\n\nAllows users to fetch additional nodes from the document store,\nbased on the prev/next relationships of the nodes.\n\nNOTE: difference with PrevNextPostprocessor is that\nthis infers forward/backwards direction.\n\nNOTE: this is a beta feature.\n\nArgs:\n    docstore (BaseDocumentStore): The document store.\n    llm_predictor (LLMPredictor): The LLM predictor.\n    num_nodes (int): The number of nodes to return (default: 1)\n    infer_prev_next_tmpl (str): The template to use for inference.\n        Required fields are {context_str} and {query_str}.",
         "type": "object",
         "properties": {
            "callback_manager": {
               "title": "Callback Manager"
            },
            "docstore": {
               "title": "Docstore"
            },
            "service_context": {
               "title": "Service Context"
            },
            "num_nodes": {
               "title": "Num Nodes",
               "default": 1,
               "type": "integer"
            },
            "infer_prev_next_tmpl": {
               "title": "Infer Prev Next Tmpl",
               "default": "The current context information is provided. \nA question is also provided. \nYou are a retrieval agent deciding whether to search the document store for additional prior context or future context. \nGiven the context and question, return PREVIOUS or NEXT or NONE. \nExamples: \n\nContext: Describes the author's experience at Y Combinator.Question: What did the author do after his time at Y Combinator? \nAnswer: NEXT \n\nContext: Describes the author's experience at Y Combinator.Question: What did the author do before his time at Y Combinator? \nAnswer: PREVIOUS \n\nContext: Describe the author's experience at Y Combinator.Question: What did the author do at Y Combinator? \nAnswer: NONE \n\nContext: {context_str}\nQuestion: {query_str}\nAnswer: ",
               "type": "string"
            },
            "refine_prev_next_tmpl": {
               "title": "Refine Prev Next Tmpl",
               "default": "The current context information is provided. \nA question is also provided. \nAn existing answer is also provided.\nYou are a retrieval agent deciding whether to search the document store for additional prior context or future context. \nGiven the context, question, and previous answer, return PREVIOUS or NEXT or NONE.\nExamples: \n\nContext: {context_msg}\nQuestion: {query_str}\nExisting Answer: {existing_answer}\nAnswer: ",
               "type": "string"
            },
            "verbose": {
               "title": "Verbose",

Node Postprocessor

               "default": false,
               "type": "boolean"
            }
         }
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "callback_manager
        (llama_index.callbacks.base.CallbackManager)"
      * "docstore
        (llama_index.storage.docstore.types.BaseDocumentStore)"
      * "infer_prev_next_tmpl (str)"
      * "num_nodes (int)"
      * "refine_prev_next_tmpl (str)"
      * "service_context
        (llama_index.indices.service_context.ServiceContext)"
      * "verbose (bool)"
   field callback_manager: CallbackManager [Optional]
   field docstore: BaseDocumentStore [Required]
   field infer_prev_next_tmpl: str = "The current context information is provided. \nA question is also provided. \nYou are a retrieval agent deciding whether to search the document store for additional prior context or future context. \nGiven the context and question, return PREVIOUS or NEXT or NONE. \nExamples: \n\nContext: Describes the author's experience at Y Combinator.Question: What did the author do after his time at Y Combinator? \nAnswer: NEXT \n\nContext: Describes the author's experience at Y Combinator.Question: What did the author do before his time at Y Combinator? \nAnswer: PREVIOUS \n\nContext: Describe the author's experience at Y Combinator.Question: What did the author do at Y Combinator? \nAnswer: NONE \n\nContext: {context_str}\nQuestion: {query_str}\nAnswer: "
   field num_nodes: int = 1
   field refine_prev_next_tmpl: str = 'The current context information is provided. \nA question is also provided. \nAn existing answer is also provided.\nYou are a retrieval agent deciding whether to search the document store for additional prior context or future context. \nGiven the context, question, and previous answer, return PREVIOUS or NEXT or NONE.\nExamples: \n\nContext: {context_msg}\nQuestion: {query_str}\nExisting Answer: {existing_answer}\nAnswer: '
   field service_context: ServiceContext [Required]
   field verbose: bool = False
   classmethod class_name() -> str
      Get the class name, used as a unique ID in serialization.
      This provides a key that makes serialization robust against
      actual class name changes.
   classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) -> Model
      Creates a new model setting __dict__ and __fields_set__ from
      trusted or pre-validated data. Default values are respected, but
      no other validation is performed. Behaves as if *Config.extra =
      'allow'* was set since it adds all passed values
   copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) -> Model
      Duplicate a model, optionally choose which fields to include,
      exclude and change.
      Parameters:
         * **include** -- fields to include in new model
         * **exclude** -- fields to exclude from new model, as with
           values this takes precedence over include
         * **update** -- values to change/add in the new model. Note:
           the data is not validated before creating the new model:
           you should trust this data
         * **deep** -- set to *True* to make a deep copy of the model
      Returns:
         new model instance
   dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) -> DictStrAny

Node Postprocessor

      Generate a dictionary representation of the model, optionally
      specifying which fields to include or exclude.
   classmethod from_dict(data: Dict[str, Any], **kwargs: Any) -> Self
   classmethod from_json(data_str: str, **kwargs: Any) -> Self
   classmethod from_orm(obj: Any) -> Model
   json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) -> unicode
      Generate a JSON representation of the model, *include* and
      *exclude* arguments as per *dict()*.
      *encoder* is an optional function to supply as *default* to
      json.dumps(), other arguments as per *json.dumps()*.
   classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   classmethod parse_obj(obj: Any) -> Model
   classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   postprocess_nodes(nodes: List[NodeWithScore], query_bundle: Optional[QueryBundle] = None) -> List[NodeWithScore]
      Postprocess nodes.
   classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') -> DictStrAny
   classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) -> unicode
   to_dict(**kwargs: Any) -> Dict[str, Any]
   to_json(**kwargs: Any) -> str
   classmethod update_forward_refs(**localns: Any) -> None
      Try to update ForwardRefs on fields based on this Model,
      globalns and localns.
   classmethod validate(value: Any) -> Model
pydantic model llama_index.indices.postprocessor.CohereRerank
      {
         "title": "CohereRerank",
         "description": "Base component object to capture class names.",
         "type": "object",
         "properties": {
            "callback_manager": {
               "title": "Callback Manager"
            },
            "model": {
               "title": "Model",
               "description": "Cohere model name.",
               "type": "string"
            },
            "top_n": {
               "title": "Top N",
               "description": "Top N nodes to return.",
               "type": "integer"
            }
         },
         "required": [
            "model",
            "top_n"
         ]
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "callback_manager
        (llama_index.callbacks.base.CallbackManager)"
      * "model (str)"
      * "top_n (int)"
   field callback_manager: CallbackManager [Optional]
   field model: str [Required]
      Cohere model name.
   field top_n: int [Required]
      Top N nodes to return.
   classmethod class_name() -> str
      Get the class name, used as a unique ID in serialization.
      This provides a key that makes serialization robust against
      actual class name changes.
   classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) -> Model

Node Postprocessor

      Creates a new model setting __dict__ and __fields_set__ from
      trusted or pre-validated data. Default values are respected, but
      no other validation is performed. Behaves as if *Config.extra =
      'allow'* was set since it adds all passed values
   copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) -> Model
      Duplicate a model, optionally choose which fields to include,
      exclude and change.
      Parameters:
         * **include** -- fields to include in new model
         * **exclude** -- fields to exclude from new model, as with
           values this takes precedence over include
         * **update** -- values to change/add in the new model. Note:
           the data is not validated before creating the new model:
           you should trust this data
         * **deep** -- set to *True* to make a deep copy of the model
      Returns:
         new model instance
   dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) -> DictStrAny
      Generate a dictionary representation of the model, optionally
      specifying which fields to include or exclude.
   classmethod from_dict(data: Dict[str, Any], **kwargs: Any) -> Self
   classmethod from_json(data_str: str, **kwargs: Any) -> Self
   classmethod from_orm(obj: Any) -> Model
   json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) -> unicode
      Generate a JSON representation of the model, *include* and
      *exclude* arguments as per *dict()*.
      *encoder* is an optional function to supply as *default* to
      json.dumps(), other arguments as per *json.dumps()*.
   classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   classmethod parse_obj(obj: Any) -> Model
   classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   postprocess_nodes(nodes: List[NodeWithScore], query_bundle: Optional[QueryBundle] = None) -> List[NodeWithScore]
      Postprocess nodes.
   classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') -> DictStrAny
   classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) -> unicode
   to_dict(**kwargs: Any) -> Dict[str, Any]
   to_json(**kwargs: Any) -> str
   classmethod update_forward_refs(**localns: Any) -> None
      Try to update ForwardRefs on fields based on this Model,

Node Postprocessor

      globalns and localns.
   classmethod validate(value: Any) -> Model
pydantic model llama_index.indices.postprocessor.EmbeddingRecencyPostprocessor
   Recency post-processor.
   This post-processor does the following steps:
   * Decides if we need to use the post-processor given the query (is
     it temporal-related?)
   * If yes, sorts nodes by date.
   * For each node, look at subsequent nodes and filter out nodes that
     have high embedding similarity with the current node. Because
     this means the subsequent node may have overlapping content with
     the current node but is also out of date
      {
         "title": "EmbeddingRecencyPostprocessor",
         "description": "Recency post-processor.\n\nThis post-processor does the following steps:\n\n- Decides if we need to use the post-processor given the query\n  (is it temporal-related?)\n- If yes, sorts nodes by date.\n- For each node, look at subsequent nodes and filter out nodes\n  that have high embedding similarity with the current node.\n  Because this means the subsequent node may have overlapping content\n  with the current node but is also out of date",
         "type": "object",
         "properties": {
            "callback_manager": {
               "title": "Callback Manager"
            },
            "service_context": {
               "title": "Service Context"
            },
            "date_key": {
               "title": "Date Key",
               "default": "date",
               "type": "string"
            },
            "similarity_cutoff": {
               "title": "Similarity Cutoff",
               "default": 0.7,
               "type": "number"
            },
            "query_embedding_tmpl": {
               "title": "Query Embedding Tmpl",
               "default": "The current document is provided.\n----------------\n{context_str}\n----------------\nGiven the document, we wish to find documents that contain \nsimilar context. Note that these documents are older than the current document, meaning that certain details may be changed. \nHowever, the high-level context should be similar.\n",
               "type": "string"
            }
         }
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "callback_manager
        (llama_index.callbacks.base.CallbackManager)"
      * "date_key (str)"
      * "query_embedding_tmpl (str)"
      * "service_context
        (llama_index.indices.service_context.ServiceContext)"
      * "similarity_cutoff (float)"
   field callback_manager: CallbackManager [Optional]
   field date_key: str = 'date'
   field query_embedding_tmpl: str = 'The current document is provided.\n----------------\n{context_str}\n----------------\nGiven the document, we wish to find documents that contain \nsimilar context. Note that these documents are older than the current document, meaning that certain details may be changed. \nHowever, the high-level context should be similar.\n'
   field service_context: ServiceContext [Required]
   field similarity_cutoff: float = 0.7
   classmethod class_name() -> str
      Get the class name, used as a unique ID in serialization.
      This provides a key that makes serialization robust against
      actual class name changes.
   classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) -> Model
      Creates a new model setting __dict__ and __fields_set__ from
      trusted or pre-validated data. Default values are respected, but
      no other validation is performed. Behaves as if *Config.extra =
      'allow'* was set since it adds all passed values
   copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) -> Model

Node Postprocessor

      Duplicate a model, optionally choose which fields to include,
      exclude and change.
      Parameters:
         * **include** -- fields to include in new model
         * **exclude** -- fields to exclude from new model, as with
           values this takes precedence over include
         * **update** -- values to change/add in the new model. Note:
           the data is not validated before creating the new model:
           you should trust this data
         * **deep** -- set to *True* to make a deep copy of the model
      Returns:
         new model instance
   dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) -> DictStrAny
      Generate a dictionary representation of the model, optionally
      specifying which fields to include or exclude.
   classmethod from_dict(data: Dict[str, Any], **kwargs: Any) -> Self
   classmethod from_json(data_str: str, **kwargs: Any) -> Self
   classmethod from_orm(obj: Any) -> Model
   json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) -> unicode
      Generate a JSON representation of the model, *include* and
      *exclude* arguments as per *dict()*.
      *encoder* is an optional function to supply as *default* to
      json.dumps(), other arguments as per *json.dumps()*.
   classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   classmethod parse_obj(obj: Any) -> Model
   classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   postprocess_nodes(nodes: List[NodeWithScore], query_bundle: Optional[QueryBundle] = None) -> List[NodeWithScore]
      Postprocess nodes.
   classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') -> DictStrAny
   classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) -> unicode
   to_dict(**kwargs: Any) -> Dict[str, Any]
   to_json(**kwargs: Any) -> str
   classmethod update_forward_refs(**localns: Any) -> None
      Try to update ForwardRefs on fields based on this Model,
      globalns and localns.
   classmethod validate(value: Any) -> Model
pydantic model llama_index.indices.postprocessor.FixedRecencyPostprocessor
   Recency post-processor.
   This post-processor does the following steps:
   * Decides if we need to use the post-processor given the query (is
     it temporal-related?)
   * If yes, sorts nodes by date.
   * Take the first k nodes (by default 1), and use that to synthesize
     an answer.
      {

Node Postprocessor

         "title": "FixedRecencyPostprocessor",
         "description": "Recency post-processor.\n\nThis post-processor does the following steps:\n\n- Decides if we need to use the post-processor given the query\n  (is it temporal-related?)\n- If yes, sorts nodes by date.\n- Take the first k nodes (by default 1), and use that to synthesize an answer.",
         "type": "object",
         "properties": {
            "callback_manager": {
               "title": "Callback Manager"
            },
            "service_context": {
               "title": "Service Context"
            },
            "top_k": {
               "title": "Top K",
               "default": 1,
               "type": "integer"
            },
            "date_key": {
               "title": "Date Key",
               "default": "date",
               "type": "string"
            }
         }
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "callback_manager
        (llama_index.callbacks.base.CallbackManager)"
      * "date_key (str)"
      * "service_context
        (llama_index.indices.service_context.ServiceContext)"
      * "top_k (int)"
   field callback_manager: CallbackManager [Optional]
   field date_key: str = 'date'
   field service_context: ServiceContext [Required]
   field top_k: int = 1
   classmethod class_name() -> str
      Get the class name, used as a unique ID in serialization.
      This provides a key that makes serialization robust against
      actual class name changes.
   classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) -> Model
      Creates a new model setting __dict__ and __fields_set__ from
      trusted or pre-validated data. Default values are respected, but
      no other validation is performed. Behaves as if *Config.extra =
      'allow'* was set since it adds all passed values
   copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) -> Model
      Duplicate a model, optionally choose which fields to include,
      exclude and change.
      Parameters:
         * **include** -- fields to include in new model
         * **exclude** -- fields to exclude from new model, as with
           values this takes precedence over include
         * **update** -- values to change/add in the new model. Note:
           the data is not validated before creating the new model:
           you should trust this data
         * **deep** -- set to *True* to make a deep copy of the model
      Returns:
         new model instance
   dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) -> DictStrAny
      Generate a dictionary representation of the model, optionally
      specifying which fields to include or exclude.
   classmethod from_dict(data: Dict[str, Any], **kwargs: Any) -> Self
   classmethod from_json(data_str: str, **kwargs: Any) -> Self
   classmethod from_orm(obj: Any) -> Model
   json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) -> unicode

Node Postprocessor

      Generate a JSON representation of the model, *include* and
      *exclude* arguments as per *dict()*.
      *encoder* is an optional function to supply as *default* to
      json.dumps(), other arguments as per *json.dumps()*.
   classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   classmethod parse_obj(obj: Any) -> Model
   classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   postprocess_nodes(nodes: List[NodeWithScore], query_bundle: Optional[QueryBundle] = None) -> List[NodeWithScore]
      Postprocess nodes.
   classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') -> DictStrAny
   classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) -> unicode
   to_dict(**kwargs: Any) -> Dict[str, Any]
   to_json(**kwargs: Any) -> str
   classmethod update_forward_refs(**localns: Any) -> None
      Try to update ForwardRefs on fields based on this Model,
      globalns and localns.
   classmethod validate(value: Any) -> Model
pydantic model llama_index.indices.postprocessor.KeywordNodePostprocessor
   Keyword-based Node processor.
      {
         "title": "KeywordNodePostprocessor",
         "description": "Keyword-based Node processor.",
         "type": "object",
         "properties": {
            "callback_manager": {
               "title": "Callback Manager"
            },
            "required_keywords": {
               "title": "Required Keywords",
               "type": "array",
               "items": {
                  "type": "string"
               }
            },
            "exclude_keywords": {
               "title": "Exclude Keywords",
               "type": "array",
               "items": {
                  "type": "string"
               }
            },
            "lang": {
               "title": "Lang",
               "default": "en",
               "type": "string"
            }
         }
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "callback_manager
        (llama_index.callbacks.base.CallbackManager)"
      * "exclude_keywords (List[str])"
      * "lang (str)"
      * "required_keywords (List[str])"
   field callback_manager: CallbackManager [Optional]
   field exclude_keywords: List[str] [Optional]
   field lang: str = 'en'
   field required_keywords: List[str] [Optional]
   classmethod class_name() -> str
      Get the class name, used as a unique ID in serialization.
      This provides a key that makes serialization robust against
      actual class name changes.
   classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) -> Model
      Creates a new model setting __dict__ and __fields_set__ from
      trusted or pre-validated data. Default values are respected, but
      no other validation is performed. Behaves as if *Config.extra =
      'allow'* was set since it adds all passed values
   copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) -> Model
      Duplicate a model, optionally choose which fields to include,
      exclude and change.

Node Postprocessor

      Parameters:
         * **include** -- fields to include in new model
         * **exclude** -- fields to exclude from new model, as with
           values this takes precedence over include
         * **update** -- values to change/add in the new model. Note:
           the data is not validated before creating the new model:
           you should trust this data
         * **deep** -- set to *True* to make a deep copy of the model
      Returns:
         new model instance
   dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) -> DictStrAny
      Generate a dictionary representation of the model, optionally
      specifying which fields to include or exclude.
   classmethod from_dict(data: Dict[str, Any], **kwargs: Any) -> Self
   classmethod from_json(data_str: str, **kwargs: Any) -> Self
   classmethod from_orm(obj: Any) -> Model
   json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) -> unicode
      Generate a JSON representation of the model, *include* and
      *exclude* arguments as per *dict()*.
      *encoder* is an optional function to supply as *default* to
      json.dumps(), other arguments as per *json.dumps()*.
   classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   classmethod parse_obj(obj: Any) -> Model
   classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   postprocess_nodes(nodes: List[NodeWithScore], query_bundle: Optional[QueryBundle] = None) -> List[NodeWithScore]
      Postprocess nodes.
   classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') -> DictStrAny
   classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) -> unicode
   to_dict(**kwargs: Any) -> Dict[str, Any]
   to_json(**kwargs: Any) -> str
   classmethod update_forward_refs(**localns: Any) -> None
      Try to update ForwardRefs on fields based on this Model,
      globalns and localns.
   classmethod validate(value: Any) -> Model
pydantic model llama_index.indices.postprocessor.LLMRerank
   LLM-based reranker.
      {
         "title": "LLMRerank",
         "description": "LLM-based reranker.",
         "type": "object",
         "properties": {
            "callback_manager": {
               "title": "Callback Manager"
            },
            "top_n": {
               "title": "Top N",
               "description": "Top N nodes to return.",
               "type": "integer"
            },
            "choice_select_prompt": {

Node Postprocessor

               "title": "Choice Select Prompt"
            },
            "choice_batch_size": {
               "title": "Choice Batch Size",
               "description": "Batch size for choice select.",
               "type": "integer"
            },
            "service_context": {
               "title": "Service Context"
            }
         },
         "required": [
            "top_n",
            "choice_batch_size"
         ]
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "callback_manager
        (llama_index.callbacks.base.CallbackManager)"
      * "choice_batch_size (int)"
      * "choice_select_prompt
        (llama_index.prompts.base.BasePromptTemplate)"
      * "service_context
        (llama_index.indices.service_context.ServiceContext)"
      * "top_n (int)"
   field callback_manager: CallbackManager [Optional]
   field choice_batch_size: int [Required]
      Batch size for choice select.
   field choice_select_prompt: BasePromptTemplate [Required]
      Choice select prompt.
   field service_context: ServiceContext [Required]
      Service context.
   field top_n: int [Required]
      Top N nodes to return.
   classmethod class_name() -> str
      Get the class name, used as a unique ID in serialization.
      This provides a key that makes serialization robust against
      actual class name changes.
   classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) -> Model
      Creates a new model setting __dict__ and __fields_set__ from
      trusted or pre-validated data. Default values are respected, but
      no other validation is performed. Behaves as if *Config.extra =
      'allow'* was set since it adds all passed values
   copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) -> Model
      Duplicate a model, optionally choose which fields to include,
      exclude and change.
      Parameters:
         * **include** -- fields to include in new model
         * **exclude** -- fields to exclude from new model, as with
           values this takes precedence over include
         * **update** -- values to change/add in the new model. Note:
           the data is not validated before creating the new model:
           you should trust this data
         * **deep** -- set to *True* to make a deep copy of the model
      Returns:
         new model instance
   dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) -> DictStrAny
      Generate a dictionary representation of the model, optionally
      specifying which fields to include or exclude.
   classmethod from_dict(data: Dict[str, Any], **kwargs: Any) -> Self
   classmethod from_json(data_str: str, **kwargs: Any) -> Self
   classmethod from_orm(obj: Any) -> Model
   json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) -> unicode

Node Postprocessor

      Generate a JSON representation of the model, *include* and
      *exclude* arguments as per *dict()*.
      *encoder* is an optional function to supply as *default* to
      json.dumps(), other arguments as per *json.dumps()*.
   classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   classmethod parse_obj(obj: Any) -> Model
   classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   postprocess_nodes(nodes: List[NodeWithScore], query_bundle: Optional[QueryBundle] = None) -> List[NodeWithScore]
      Postprocess nodes.
   classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') -> DictStrAny
   classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) -> unicode
   to_dict(**kwargs: Any) -> Dict[str, Any]
   to_json(**kwargs: Any) -> str
   classmethod update_forward_refs(**localns: Any) -> None
      Try to update ForwardRefs on fields based on this Model,
      globalns and localns.
   classmethod validate(value: Any) -> Model
pydantic model llama_index.indices.postprocessor.LongContextReorder
   Models struggle to access significant details found in the center
   of extended contexts. A study (https://arxiv.org/abs/2307.03172)
   observed that the best performance typically arises when crucial
   data is positioned at the start or conclusion of the input context.
   Additionally, as the input context lengthens, performance drops
   notably, even in models designed for long contexts.".
      {
         "title": "LongContextReorder",
         "description": "Models struggle to access significant details found\nin the center of extended contexts. A study\n(https://arxiv.org/abs/2307.03172) observed that the best\nperformance typically arises when crucial data is positioned\nat the start or conclusion of the input context. Additionally,\nas the input context lengthens, performance drops notably, even\nin models designed for long contexts.\".",
         "type": "object",
         "properties": {
            "callback_manager": {
               "title": "Callback Manager"
            }
         }
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "callback_manager
        (llama_index.callbacks.base.CallbackManager)"
   field callback_manager: CallbackManager [Optional]
   classmethod class_name() -> str
      Get the class name, used as a unique ID in serialization.
      This provides a key that makes serialization robust against
      actual class name changes.
   classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) -> Model
      Creates a new model setting __dict__ and __fields_set__ from
      trusted or pre-validated data. Default values are respected, but
      no other validation is performed. Behaves as if *Config.extra =
      'allow'* was set since it adds all passed values
   copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) -> Model
      Duplicate a model, optionally choose which fields to include,
      exclude and change.
      Parameters:
         * **include** -- fields to include in new model

Node Postprocessor

         * **exclude** -- fields to exclude from new model, as with
           values this takes precedence over include
         * **update** -- values to change/add in the new model. Note:
           the data is not validated before creating the new model:
           you should trust this data
         * **deep** -- set to *True* to make a deep copy of the model
      Returns:
         new model instance
   dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) -> DictStrAny
      Generate a dictionary representation of the model, optionally
      specifying which fields to include or exclude.
   classmethod from_dict(data: Dict[str, Any], **kwargs: Any) -> Self
   classmethod from_json(data_str: str, **kwargs: Any) -> Self
   classmethod from_orm(obj: Any) -> Model
   json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) -> unicode
      Generate a JSON representation of the model, *include* and
      *exclude* arguments as per *dict()*.
      *encoder* is an optional function to supply as *default* to
      json.dumps(), other arguments as per *json.dumps()*.
   classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   classmethod parse_obj(obj: Any) -> Model
   classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   postprocess_nodes(nodes: List[NodeWithScore], query_bundle: Optional[QueryBundle] = None) -> List[NodeWithScore]
      Postprocess nodes.
   classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') -> DictStrAny
   classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) -> unicode
   to_dict(**kwargs: Any) -> Dict[str, Any]
   to_json(**kwargs: Any) -> str
   classmethod update_forward_refs(**localns: Any) -> None
      Try to update ForwardRefs on fields based on this Model,
      globalns and localns.
   classmethod validate(value: Any) -> Model
pydantic model llama_index.indices.postprocessor.MetadataReplacementPostProcessor
      {
         "title": "MetadataReplacementPostProcessor",
         "description": "Base component object to capture class names.",
         "type": "object",
         "properties": {
            "callback_manager": {
               "title": "Callback Manager"
            },
            "target_metadata_key": {
               "title": "Target Metadata Key",
               "description": "Target metadata key to replace node content with.",
               "type": "string"
            }
         },
         "required": [
            "target_metadata_key"
         ]
      }
   Config:

Node Postprocessor

      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "callback_manager
        (llama_index.callbacks.base.CallbackManager)"
      * "target_metadata_key (str)"
   field callback_manager: CallbackManager [Optional]
   field target_metadata_key: str [Required]
      Target metadata key to replace node content with.
   classmethod class_name() -> str
      Get the class name, used as a unique ID in serialization.
      This provides a key that makes serialization robust against
      actual class name changes.
   classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) -> Model
      Creates a new model setting __dict__ and __fields_set__ from
      trusted or pre-validated data. Default values are respected, but
      no other validation is performed. Behaves as if *Config.extra =
      'allow'* was set since it adds all passed values
   copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) -> Model
      Duplicate a model, optionally choose which fields to include,
      exclude and change.
      Parameters:
         * **include** -- fields to include in new model
         * **exclude** -- fields to exclude from new model, as with
           values this takes precedence over include
         * **update** -- values to change/add in the new model. Note:
           the data is not validated before creating the new model:
           you should trust this data
         * **deep** -- set to *True* to make a deep copy of the model
      Returns:
         new model instance
   dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) -> DictStrAny
      Generate a dictionary representation of the model, optionally
      specifying which fields to include or exclude.
   classmethod from_dict(data: Dict[str, Any], **kwargs: Any) -> Self
   classmethod from_json(data_str: str, **kwargs: Any) -> Self
   classmethod from_orm(obj: Any) -> Model
   json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) -> unicode
      Generate a JSON representation of the model, *include* and
      *exclude* arguments as per *dict()*.
      *encoder* is an optional function to supply as *default* to
      json.dumps(), other arguments as per *json.dumps()*.
   classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   classmethod parse_obj(obj: Any) -> Model
   classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   postprocess_nodes(nodes: List[NodeWithScore], query_bundle: Optional[QueryBundle] = None) -> List[NodeWithScore]

Node Postprocessor

      Postprocess nodes.
   classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') -> DictStrAny
   classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) -> unicode
   to_dict(**kwargs: Any) -> Dict[str, Any]
   to_json(**kwargs: Any) -> str
   classmethod update_forward_refs(**localns: Any) -> None
      Try to update ForwardRefs on fields based on this Model,
      globalns and localns.
   classmethod validate(value: Any) -> Model
pydantic model llama_index.indices.postprocessor.NERPIINodePostprocessor
   NER PII Node processor.
   Uses a HF transformers model.
      {
         "title": "NERPIINodePostprocessor",
         "description": "NER PII Node processor.\n\nUses a HF transformers model.",
         "type": "object",
         "properties": {
            "callback_manager": {
               "title": "Callback Manager"
            },
            "pii_node_info_key": {
               "title": "Pii Node Info Key",
               "default": "__pii_node_info__",
               "type": "string"
            }
         }
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "callback_manager
        (llama_index.callbacks.base.CallbackManager)"
      * "pii_node_info_key (str)"
   field callback_manager: CallbackManager [Optional]
   field pii_node_info_key: str = '__pii_node_info__'
   classmethod class_name() -> str
      Get the class name, used as a unique ID in serialization.
      This provides a key that makes serialization robust against
      actual class name changes.
   classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) -> Model
      Creates a new model setting __dict__ and __fields_set__ from
      trusted or pre-validated data. Default values are respected, but
      no other validation is performed. Behaves as if *Config.extra =
      'allow'* was set since it adds all passed values
   copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) -> Model
      Duplicate a model, optionally choose which fields to include,
      exclude and change.
      Parameters:
         * **include** -- fields to include in new model
         * **exclude** -- fields to exclude from new model, as with
           values this takes precedence over include
         * **update** -- values to change/add in the new model. Note:
           the data is not validated before creating the new model:
           you should trust this data
         * **deep** -- set to *True* to make a deep copy of the model
      Returns:
         new model instance
   dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) -> DictStrAny
      Generate a dictionary representation of the model, optionally
      specifying which fields to include or exclude.
   classmethod from_dict(data: Dict[str, Any], **kwargs: Any) -> Self
   classmethod from_json(data_str: str, **kwargs: Any) -> Self
   classmethod from_orm(obj: Any) -> Model

Node Postprocessor

   json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) -> unicode
      Generate a JSON representation of the model, *include* and
      *exclude* arguments as per *dict()*.
      *encoder* is an optional function to supply as *default* to
      json.dumps(), other arguments as per *json.dumps()*.
   mask_pii(ner: Callable, text: str) -> Tuple[str, Dict]
      Mask PII in text.
   classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   classmethod parse_obj(obj: Any) -> Model
   classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   postprocess_nodes(nodes: List[NodeWithScore], query_bundle: Optional[QueryBundle] = None) -> List[NodeWithScore]
      Postprocess nodes.
   classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') -> DictStrAny
   classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) -> unicode
   to_dict(**kwargs: Any) -> Dict[str, Any]
   to_json(**kwargs: Any) -> str
   classmethod update_forward_refs(**localns: Any) -> None
      Try to update ForwardRefs on fields based on this Model,
      globalns and localns.
   classmethod validate(value: Any) -> Model
pydantic model llama_index.indices.postprocessor.PIINodePostprocessor
   PII Node processor.
   NOTE: the ServiceContext should contain a LOCAL model, not an
   external API.
   NOTE: this is a beta feature, the API might change.
   Parameters:
      **service_context** (*ServiceContext*) -- Service context.
      {
         "title": "PIINodePostprocessor",
         "description": "PII Node processor.\n\nNOTE: the ServiceContext should contain a LOCAL model, not an external API.\n\nNOTE: this is a beta feature, the API might change.\n\nArgs:\n    service_context (ServiceContext): Service context.",
         "type": "object",
         "properties": {
            "callback_manager": {
               "title": "Callback Manager"
            },
            "service_context": {
               "title": "Service Context"
            },
            "pii_str_tmpl": {
               "title": "Pii Str Tmpl",
               "default": "The current context information is provided. \nA task is also provided to mask the PII within the context. \nReturn the text, with all PII masked out, and a mapping of the original PII to the masked PII. \nReturn the output of the task in JSON. \nContext:\nHello Zhang Wei, I am John. Your AnyCompany Financial Services, LLC credit card account 1111-0000-1111-0008 has a minimum payment of $24.53 that is due by July 31st. Based on your autopay settings, we will withdraw your payment. Task: Mask out the PII, replace each PII with a tag, and return the text. Return the mapping in JSON. \nOutput: \nHello [NAME1], I am [NAME2]. Your AnyCompany Financial Services, LLC credit card account [CREDIT_CARD_NUMBER] has a minimum payment of $24.53 that is due by [DATE_TIME]. Based on your autopay settings, we will withdraw your payment. Output Mapping:\n{{\"NAME1\": \"Zhang Wei\", \"NAME2\": \"John\", \"CREDIT_CARD_NUMBER\": \"1111-0000-1111-0008\", \"DATE_TIME\": \"July 31st\"}}\nContext:\n{context_str}\nTask: {query_str}\nOutput: \n",

Node Postprocessor

               "type": "string"
            },
            "pii_node_info_key": {
               "title": "Pii Node Info Key",
               "default": "__pii_node_info__",
               "type": "string"
            }
         }
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "callback_manager
        (llama_index.callbacks.base.CallbackManager)"
      * "pii_node_info_key (str)"
      * "pii_str_tmpl (str)"
      * "service_context
        (llama_index.indices.service_context.ServiceContext)"
   field callback_manager: CallbackManager [Optional]
   field pii_node_info_key: str = '__pii_node_info__'
   field pii_str_tmpl: str = 'The current context information is provided. \nA task is also provided to mask the PII within the context. \nReturn the text, with all PII masked out, and a mapping of the original PII to the masked PII. \nReturn the output of the task in JSON. \nContext:\nHello Zhang Wei, I am John. Your AnyCompany Financial Services, LLC credit card account 1111-0000-1111-0008 has a minimum payment of $24.53 that is due by July 31st. Based on your autopay settings, we will withdraw your payment. Task: Mask out the PII, replace each PII with a tag, and return the text. Return the mapping in JSON. \nOutput: \nHello [NAME1], I am [NAME2]. Your AnyCompany Financial Services, LLC credit card account [CREDIT_CARD_NUMBER] has a minimum payment of $24.53 that is due by [DATE_TIME]. Based on your autopay settings, we will withdraw your payment. Output Mapping:\n{{"NAME1": "Zhang Wei", "NAME2": "John", "CREDIT_CARD_NUMBER": "1111-0000-1111-0008", "DATE_TIME": "July 31st"}}\nContext:\n{context_str}\nTask: {query_str}\nOutput: \n'
   field service_context: ServiceContext [Required]
   classmethod class_name() -> str
      Get the class name, used as a unique ID in serialization.
      This provides a key that makes serialization robust against
      actual class name changes.
   classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) -> Model
      Creates a new model setting __dict__ and __fields_set__ from
      trusted or pre-validated data. Default values are respected, but
      no other validation is performed. Behaves as if *Config.extra =
      'allow'* was set since it adds all passed values
   copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) -> Model
      Duplicate a model, optionally choose which fields to include,
      exclude and change.
      Parameters:
         * **include** -- fields to include in new model
         * **exclude** -- fields to exclude from new model, as with
           values this takes precedence over include
         * **update** -- values to change/add in the new model. Note:
           the data is not validated before creating the new model:
           you should trust this data
         * **deep** -- set to *True* to make a deep copy of the model
      Returns:
         new model instance
   dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) -> DictStrAny

Node Postprocessor

      Generate a dictionary representation of the model, optionally
      specifying which fields to include or exclude.
   classmethod from_dict(data: Dict[str, Any], **kwargs: Any) -> Self
   classmethod from_json(data_str: str, **kwargs: Any) -> Self
   classmethod from_orm(obj: Any) -> Model
   json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) -> unicode
      Generate a JSON representation of the model, *include* and
      *exclude* arguments as per *dict()*.
      *encoder* is an optional function to supply as *default* to
      json.dumps(), other arguments as per *json.dumps()*.
   mask_pii(text: str) -> Tuple[str, Dict]
      Mask PII in text.
   classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   classmethod parse_obj(obj: Any) -> Model
   classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   postprocess_nodes(nodes: List[NodeWithScore], query_bundle: Optional[QueryBundle] = None) -> List[NodeWithScore]
      Postprocess nodes.
   classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') -> DictStrAny
   classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) -> unicode
   to_dict(**kwargs: Any) -> Dict[str, Any]
   to_json(**kwargs: Any) -> str
   classmethod update_forward_refs(**localns: Any) -> None
      Try to update ForwardRefs on fields based on this Model,
      globalns and localns.
   classmethod validate(value: Any) -> Model
pydantic model llama_index.indices.postprocessor.PrevNextNodePostprocessor
   Previous/Next Node post-processor.
   Allows users to fetch additional nodes from the document store,
   based on the relationships of the nodes.
   NOTE: this is a beta feature.
   Parameters:
      * **docstore** (*BaseDocumentStore*) -- The document store.
      * **num_nodes** (*int*) -- The number of nodes to return
        (default: 1)
      * **mode** (*str*) -- The mode of the post-processor. Can be
        "previous", "next", or "both.
      {
         "title": "PrevNextNodePostprocessor",
         "description": "Previous/Next Node post-processor.\n\nAllows users to fetch additional nodes from the document store,\nbased on the relationships of the nodes.\n\nNOTE: this is a beta feature.\n\nArgs:\n    docstore (BaseDocumentStore): The document store.\n    num_nodes (int): The number of nodes to return (default: 1)\n    mode (str): The mode of the post-processor.\n        Can be \"previous\", \"next\", or \"both.",
         "type": "object",
         "properties": {
            "callback_manager": {
               "title": "Callback Manager"

Node Postprocessor

            },
            "docstore": {
               "title": "Docstore"
            },
            "num_nodes": {
               "title": "Num Nodes",
               "default": 1,
               "type": "integer"
            },
            "mode": {
               "title": "Mode",
               "default": "next",
               "type": "string"
            }
         }
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "callback_manager
        (llama_index.callbacks.base.CallbackManager)"
      * "docstore
        (llama_index.storage.docstore.types.BaseDocumentStore)"
      * "mode (str)"
      * "num_nodes (int)"
   Validators:
      * "_validate_mode" » "mode"
   field callback_manager: CallbackManager [Optional]
   field docstore: BaseDocumentStore [Required]
   field mode: str = 'next'
      Validated by:
         * "_validate_mode"
   field num_nodes: int = 1
   classmethod class_name() -> str
      Get the class name, used as a unique ID in serialization.
      This provides a key that makes serialization robust against
      actual class name changes.
   classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) -> Model
      Creates a new model setting __dict__ and __fields_set__ from
      trusted or pre-validated data. Default values are respected, but
      no other validation is performed. Behaves as if *Config.extra =
      'allow'* was set since it adds all passed values
   copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) -> Model
      Duplicate a model, optionally choose which fields to include,
      exclude and change.
      Parameters:
         * **include** -- fields to include in new model
         * **exclude** -- fields to exclude from new model, as with
           values this takes precedence over include
         * **update** -- values to change/add in the new model. Note:
           the data is not validated before creating the new model:
           you should trust this data
         * **deep** -- set to *True* to make a deep copy of the model
      Returns:
         new model instance
   dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) -> DictStrAny
      Generate a dictionary representation of the model, optionally
      specifying which fields to include or exclude.
   classmethod from_dict(data: Dict[str, Any], **kwargs: Any) -> Self
   classmethod from_json(data_str: str, **kwargs: Any) -> Self
   classmethod from_orm(obj: Any) -> Model
   json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) -> unicode
      Generate a JSON representation of the model, *include* and
      *exclude* arguments as per *dict()*.

Node Postprocessor

      *encoder* is an optional function to supply as *default* to
      json.dumps(), other arguments as per *json.dumps()*.
   classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   classmethod parse_obj(obj: Any) -> Model
   classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   postprocess_nodes(nodes: List[NodeWithScore], query_bundle: Optional[QueryBundle] = None) -> List[NodeWithScore]
      Postprocess nodes.
   classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') -> DictStrAny
   classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) -> unicode
   to_dict(**kwargs: Any) -> Dict[str, Any]
   to_json(**kwargs: Any) -> str
   classmethod update_forward_refs(**localns: Any) -> None
      Try to update ForwardRefs on fields based on this Model,
      globalns and localns.
   classmethod validate(value: Any) -> Model
pydantic model llama_index.indices.postprocessor.SentenceEmbeddingOptimizer
   Optimization of a text chunk given the query by shortening the
   input text.
      {
         "title": "SentenceEmbeddingOptimizer",
         "description": "Optimization of a text chunk given the query by shortening the input text.",
         "type": "object",
         "properties": {
            "callback_manager": {
               "title": "Callback Manager"
            },
            "percentile_cutoff": {
               "title": "Percentile Cutoff",
               "description": "Percentile cutoff for the top k sentences to use.",
               "type": "number"
            },
            "threshold_cutoff": {
               "title": "Threshold Cutoff",
               "description": "Threshold cutoff for similarity for each sentence to use.",
               "type": "number"
            },
            "context_before": {
               "title": "Context Before",
               "description": "Number of sentences before retrieved sentence for further context",
               "type": "integer"
            },
            "context_after": {
               "title": "Context After",
               "description": "Number of sentences after retrieved sentence for further context",
               "type": "integer"
            }
         }
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "callback_manager
        (llama_index.callbacks.base.CallbackManager)"
      * "context_after (Optional[int])"
      * "context_before (Optional[int])"
      * "percentile_cutoff (Optional[float])"
      * "threshold_cutoff (Optional[float])"
   field callback_manager: CallbackManager [Optional]
   field context_after: Optional[int] = None
      Number of sentences after retrieved sentence for further context
   field context_before: Optional[int] = None
      Number of sentences before retrieved sentence for further
      context
   field percentile_cutoff: Optional[float] = None
      Percentile cutoff for the top k sentences to use.
   field threshold_cutoff: Optional[float] = None
      Threshold cutoff for similarity for each sentence to use.
   classmethod class_name() -> str
      Get the class name, used as a unique ID in serialization.
      This provides a key that makes serialization robust against
      actual class name changes.
   classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) -> Model
      Creates a new model setting __dict__ and __fields_set__ from

Node Postprocessor

      trusted or pre-validated data. Default values are respected, but
      no other validation is performed. Behaves as if *Config.extra =
      'allow'* was set since it adds all passed values
   copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) -> Model
      Duplicate a model, optionally choose which fields to include,
      exclude and change.
      Parameters:
         * **include** -- fields to include in new model
         * **exclude** -- fields to exclude from new model, as with
           values this takes precedence over include
         * **update** -- values to change/add in the new model. Note:
           the data is not validated before creating the new model:
           you should trust this data
         * **deep** -- set to *True* to make a deep copy of the model
      Returns:
         new model instance
   dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) -> DictStrAny
      Generate a dictionary representation of the model, optionally
      specifying which fields to include or exclude.
   classmethod from_dict(data: Dict[str, Any], **kwargs: Any) -> Self
   classmethod from_json(data_str: str, **kwargs: Any) -> Self
   classmethod from_orm(obj: Any) -> Model
   json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) -> unicode
      Generate a JSON representation of the model, *include* and
      *exclude* arguments as per *dict()*.
      *encoder* is an optional function to supply as *default* to
      json.dumps(), other arguments as per *json.dumps()*.
   classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   classmethod parse_obj(obj: Any) -> Model
   classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   postprocess_nodes(nodes: List[NodeWithScore], query_bundle: Optional[QueryBundle] = None) -> List[NodeWithScore]
      Optimize a node text given the query by shortening the node
      text.
   classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') -> DictStrAny
   classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) -> unicode
   to_dict(**kwargs: Any) -> Dict[str, Any]
   to_json(**kwargs: Any) -> str
   classmethod update_forward_refs(**localns: Any) -> None
      Try to update ForwardRefs on fields based on this Model,

Node Postprocessor

      globalns and localns.
   classmethod validate(value: Any) -> Model
pydantic model llama_index.indices.postprocessor.SentenceTransformerRerank
      {
         "title": "SentenceTransformerRerank",
         "description": "Base component object to capture class names.",
         "type": "object",
         "properties": {
            "callback_manager": {
               "title": "Callback Manager"
            },
            "model": {
               "title": "Model",
               "ddescription": "Sentence transformer model name.",
               "type": "string"
            },
            "top_n": {
               "title": "Top N",
               "description": "Number of nodes to return sorted by score.",
               "type": "integer"
            }
         },
         "required": [
            "model",
            "top_n"
         ]
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "callback_manager
        (llama_index.callbacks.base.CallbackManager)"
      * "model (str)"
      * "top_n (int)"
   field callback_manager: CallbackManager [Optional]
   field model: str [Required]
   field top_n: int [Required]
      Number of nodes to return sorted by score.
   classmethod class_name() -> str
      Get the class name, used as a unique ID in serialization.
      This provides a key that makes serialization robust against
      actual class name changes.
   classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) -> Model
      Creates a new model setting __dict__ and __fields_set__ from
      trusted or pre-validated data. Default values are respected, but
      no other validation is performed. Behaves as if *Config.extra =
      'allow'* was set since it adds all passed values
   copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) -> Model
      Duplicate a model, optionally choose which fields to include,
      exclude and change.
      Parameters:
         * **include** -- fields to include in new model
         * **exclude** -- fields to exclude from new model, as with
           values this takes precedence over include
         * **update** -- values to change/add in the new model. Note:
           the data is not validated before creating the new model:
           you should trust this data
         * **deep** -- set to *True* to make a deep copy of the model
      Returns:
         new model instance
   dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) -> DictStrAny
      Generate a dictionary representation of the model, optionally
      specifying which fields to include or exclude.
   classmethod from_dict(data: Dict[str, Any], **kwargs: Any) -> Self
   classmethod from_json(data_str: str, **kwargs: Any) -> Self
   classmethod from_orm(obj: Any) -> Model
   json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) -> unicode

Node Postprocessor

      Generate a JSON representation of the model, *include* and
      *exclude* arguments as per *dict()*.
      *encoder* is an optional function to supply as *default* to
      json.dumps(), other arguments as per *json.dumps()*.
   classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   classmethod parse_obj(obj: Any) -> Model
   classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   postprocess_nodes(nodes: List[NodeWithScore], query_bundle: Optional[QueryBundle] = None) -> List[NodeWithScore]
      Postprocess nodes.
   classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') -> DictStrAny
   classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) -> unicode
   to_dict(**kwargs: Any) -> Dict[str, Any]
   to_json(**kwargs: Any) -> str
   classmethod update_forward_refs(**localns: Any) -> None
      Try to update ForwardRefs on fields based on this Model,
      globalns and localns.
   classmethod validate(value: Any) -> Model
pydantic model llama_index.indices.postprocessor.SimilarityPostprocessor
   Similarity-based Node processor.
      {
         "title": "SimilarityPostprocessor",
         "description": "Similarity-based Node processor.",
         "type": "object",
         "properties": {
            "callback_manager": {
               "title": "Callback Manager"
            },
            "similarity_cutoff": {
               "title": "Similarity Cutoff",
               "type": "number"
            }
         }
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "callback_manager
        (llama_index.callbacks.base.CallbackManager)"
      * "similarity_cutoff (float)"
   field callback_manager: CallbackManager [Optional]
   field similarity_cutoff: float = None
   classmethod class_name() -> str
      Get the class name, used as a unique ID in serialization.
      This provides a key that makes serialization robust against
      actual class name changes.
   classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) -> Model
      Creates a new model setting __dict__ and __fields_set__ from
      trusted or pre-validated data. Default values are respected, but
      no other validation is performed. Behaves as if *Config.extra =
      'allow'* was set since it adds all passed values
   copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) -> Model
      Duplicate a model, optionally choose which fields to include,
      exclude and change.
      Parameters:
         * **include** -- fields to include in new model
         * **exclude** -- fields to exclude from new model, as with
           values this takes precedence over include
         * **update** -- values to change/add in the new model. Note:
           the data is not validated before creating the new model:
           you should trust this data
         * **deep** -- set to *True* to make a deep copy of the model
      Returns:
         new model instance
   dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) -> DictStrAny

Node Postprocessor

      Generate a dictionary representation of the model, optionally
      specifying which fields to include or exclude.
   classmethod from_dict(data: Dict[str, Any], **kwargs: Any) -> Self
   classmethod from_json(data_str: str, **kwargs: Any) -> Self
   classmethod from_orm(obj: Any) -> Model
   json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) -> unicode
      Generate a JSON representation of the model, *include* and
      *exclude* arguments as per *dict()*.
      *encoder* is an optional function to supply as *default* to
      json.dumps(), other arguments as per *json.dumps()*.
   classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   classmethod parse_obj(obj: Any) -> Model
   classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   postprocess_nodes(nodes: List[NodeWithScore], query_bundle: Optional[QueryBundle] = None) -> List[NodeWithScore]
      Postprocess nodes.
   classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') -> DictStrAny
   classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) -> unicode
   to_dict(**kwargs: Any) -> Dict[str, Any]
   to_json(**kwargs: Any) -> str
   classmethod update_forward_refs(**localns: Any) -> None
      Try to update ForwardRefs on fields based on this Model,
      globalns and localns.
   classmethod validate(value: Any) -> Model
pydantic model llama_index.indices.postprocessor.TimeWeightedPostprocessor
   Time-weighted post-processor.
   Reranks a set of nodes based on their recency.
      {
         "title": "TimeWeightedPostprocessor",
         "description": "Time-weighted post-processor.\n\nReranks a set of nodes based on their recency.",
         "type": "object",
         "properties": {
            "callback_manager": {
               "title": "Callback Manager"
            },
            "time_decay": {
               "title": "Time Decay",
               "default": 0.99,
               "type": "number"
            },
            "last_accessed_key": {
               "title": "Last Accessed Key",
               "default": "__last_accessed__",
               "type": "string"
            },
            "time_access_refresh": {
               "title": "Time Access Refresh",
               "default": true,
               "type": "boolean"
            },
            "now": {
               "title": "Now",
               "type": "number"
            },
            "top_k": {
               "title": "Top K",
               "default": 1,
               "type": "integer"
            }
         }
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "callback_manager
        (llama_index.callbacks.base.CallbackManager)"
      * "last_accessed_key (str)"

Node Postprocessor

      * "now (Optional[float])"
      * "time_access_refresh (bool)"
      * "time_decay (float)"
      * "top_k (int)"
   field callback_manager: CallbackManager [Optional]
   field last_accessed_key: str = '__last_accessed__'
   field now: Optional[float] = None
   field time_access_refresh: bool = True
   field time_decay: float = 0.99
   field top_k: int = 1
   classmethod class_name() -> str
      Get the class name, used as a unique ID in serialization.
      This provides a key that makes serialization robust against
      actual class name changes.
   classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) -> Model
      Creates a new model setting __dict__ and __fields_set__ from
      trusted or pre-validated data. Default values are respected, but
      no other validation is performed. Behaves as if *Config.extra =
      'allow'* was set since it adds all passed values
   copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) -> Model
      Duplicate a model, optionally choose which fields to include,
      exclude and change.
      Parameters:
         * **include** -- fields to include in new model
         * **exclude** -- fields to exclude from new model, as with
           values this takes precedence over include
         * **update** -- values to change/add in the new model. Note:
           the data is not validated before creating the new model:
           you should trust this data
         * **deep** -- set to *True* to make a deep copy of the model
      Returns:
         new model instance
   dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) -> DictStrAny
      Generate a dictionary representation of the model, optionally
      specifying which fields to include or exclude.
   classmethod from_dict(data: Dict[str, Any], **kwargs: Any) -> Self
   classmethod from_json(data_str: str, **kwargs: Any) -> Self
   classmethod from_orm(obj: Any) -> Model
   json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) -> unicode
      Generate a JSON representation of the model, *include* and
      *exclude* arguments as per *dict()*.
      *encoder* is an optional function to supply as *default* to
      json.dumps(), other arguments as per *json.dumps()*.
   classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   classmethod parse_obj(obj: Any) -> Model
   classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model

Node Postprocessor


Finetuning

Finetuning modules.
class llama_index.finetuning.EmbeddingAdapterFinetuneEngine(dataset: EmbeddingQAFinetuneDataset, embed_model: BaseEmbedding, batch_size: int = 10, epochs: int = 1, adapter_model: Optional[Any] = None, dim: Optional[int] = None, device: Optional[str] = None, model_output_path: str = 'model_output', model_checkpoint_path: Optional[str] = None, checkpoint_save_steps: int = 100, verbose: bool = False, bias: bool = False, **train_kwargs: Any)
   Embedding adapter finetune engine.
   Parameters:
      * **dataset** (*EmbeddingQAFinetuneDataset*) -- Dataset to
        finetune on.
      * **embed_model** (*BaseEmbedding*) -- Embedding model to
        finetune.
      * **batch_size** (*Optional**[**int**]*) -- Batch size. Defaults
        to 10.
      * **epochs** (*Optional**[**int**]*) -- Number of epochs.
        Defaults to 1.
      * **dim** (*Optional**[**int**]*) -- Dimension of embedding.
        Defaults to None.
      * **adapter_model** (*Optional**[**BaseAdapter**]*) -- Adapter
        model. Defaults to None, in which case a linear adapter is
        used.
      * **device** (*Optional**[**str**]*) -- Device to use. Defaults
        to None.
      * **model_output_path** (*str*) -- Path to save model output.
        Defaults to "model_output".
      * **model_checkpoint_path** (*Optional**[**str**]*) -- Path to
        save model checkpoints. Defaults to None (don't save
        checkpoints).
      * **verbose** (*bool*) -- Whether to show progress bar. Defaults
        to False.
      * **bias** (*bool*) -- Whether to use bias. Defaults to False.
   finetune(**train_kwargs: Any) -> None
      Finetune.
   classmethod from_model_path(dataset: EmbeddingQAFinetuneDataset, embed_model: BaseEmbedding, model_path: str, model_cls: Optional[Type[Any]] = None, **kwargs: Any) -> EmbeddingAdapterFinetuneEngine
      Load from model path.
      Parameters:
         * **dataset** (*EmbeddingQAFinetuneDataset*) -- Dataset to
           finetune on.
         * **embed_model** (*BaseEmbedding*) -- Embedding model to
           finetune.
         * **model_path** (*str*) -- Path to model.
         * **model_cls** (*Optional**[**Type**[**Any**]**]*) --
           Adapter model class. Defaults to None.
         * ****kwargs** (*Any*) -- Additional kwargs (see __init__)
   get_finetuned_model(**model_kwargs: Any) -> BaseEmbedding
      Get finetuned model.
   smart_batching_collate(batch: List) -> Tuple[Any, Any]
      Smart batching collate.
pydantic model llama_index.finetuning.EmbeddingQAFinetuneDataset
   Embedding QA Finetuning Dataset.
   Parameters:
      * **queries** (*Dict**[**str**, **str**]*) -- Dict id -> query.
      * **corpus** (*Dict**[**str**, **str**]*) -- Dict id -> string.
      * **relevant_docs** (*Dict**[**str**, **List**[**str**]**]*) --
        Dict query id -> list of doc ids.
      {
         "title": "EmbeddingQAFinetuneDataset",
         "description": "Embedding QA Finetuning Dataset.\n\nArgs:\n    queries (Dict[str, str]): Dict id -> query.\n    corpus (Dict[str, str]): Dict id -> string.\n    relevant_docs (Dict[str, List[str]]): Dict query id -> list of doc ids.",

Finetuning

         "type": "object",
         "properties": {
            "queries": {
               "title": "Queries",
               "type": "object",
               "additionalProperties": {
                  "type": "string"
               }
            },
            "corpus": {
               "title": "Corpus",
               "type": "object",
               "additionalProperties": {
                  "type": "string"
               }
            },
            "relevant_docs": {
               "title": "Relevant Docs",
               "type": "object",
               "additionalProperties": {
                  "type": "array",
                  "items": {
                     "type": "string"
                  }
               }
            }
         },
         "required": [
            "queries",
            "corpus",
            "relevant_docs"
         ]
      }
   Fields:
      * "corpus (Dict[str, str])"
      * "queries (Dict[str, str])"
      * "relevant_docs (Dict[str, List[str]])"
   field corpus: Dict[str, str] [Required]
   field queries: Dict[str, str] [Required]
   field relevant_docs: Dict[str, List[str]] [Required]
   classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) -> Model
      Creates a new model setting __dict__ and __fields_set__ from
      trusted or pre-validated data. Default values are respected, but
      no other validation is performed. Behaves as if *Config.extra =
      'allow'* was set since it adds all passed values
   copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) -> Model
      Duplicate a model, optionally choose which fields to include,
      exclude and change.
      Parameters:
         * **include** -- fields to include in new model
         * **exclude** -- fields to exclude from new model, as with
           values this takes precedence over include
         * **update** -- values to change/add in the new model. Note:
           the data is not validated before creating the new model:
           you should trust this data
         * **deep** -- set to *True* to make a deep copy of the model
      Returns:
         new model instance
   dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) -> DictStrAny
      Generate a dictionary representation of the model, optionally
      specifying which fields to include or exclude.
   classmethod from_json(path: str) -> EmbeddingQAFinetuneDataset
      Load json.
   classmethod from_orm(obj: Any) -> Model
   json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) -> unicode
      Generate a JSON representation of the model, *include* and
      *exclude* arguments as per *dict()*.
      *encoder* is an optional function to supply as *default* to
      json.dumps(), other arguments as per *json.dumps()*.

Finetuning

   classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   classmethod parse_obj(obj: Any) -> Model
   classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) -> Model
   save_json(path: str) -> None
      Save json.
   classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') -> DictStrAny
   classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) -> unicode
   classmethod update_forward_refs(**localns: Any) -> None
      Try to update ForwardRefs on fields based on this Model,
      globalns and localns.
   classmethod validate(value: Any) -> Model
class llama_index.finetuning.OpenAIFinetuneEngine(base_model: str, data_path: str, verbose: bool = False, start_job_id: Optional[str] = None, validate_json: bool = True)
   OpenAI Finetuning Engine.
   finetune() -> None
      Finetune model.
   classmethod from_finetuning_handler(finetuning_handler: OpenAIFineTuningHandler, base_model: str, data_path: str, **kwargs: Any) -> OpenAIFinetuneEngine
      Initialize from finetuning handler.
      Used to finetune an OpenAI model into another OpenAI model (e.g.
      gpt-3.5-turbo on top of GPT-4).
   get_current_job() -> Any
      Get current job.
   get_finetuned_model(**model_kwargs: Any) -> LLM
      Gets finetuned model.
class llama_index.finetuning.SentenceTransformersFinetuneEngine(dataset: EmbeddingQAFinetuneDataset, model_id: str = 'BAAI/bge-small-en', model_output_path: str = 'exp_finetune', batch_size: int = 10, val_dataset: Optional[EmbeddingQAFinetuneDataset] = None, loss: Optional[Any] = None, epochs: int = 2, show_progress_bar: bool = True, evaluation_steps: int = 50)
   Sentence Transformers Finetune Engine.
   finetune(**train_kwargs: Any) -> None
      Finetune model.
   get_finetuned_model(**model_kwargs: Any) -> BaseEmbedding
      Gets finetuned model.
llama_index.finetuning.generate_qa_embedding_pairs(nodes: List[TextNode], llm: Optional[LLM] = None, qa_generate_prompt_tmpl: str = 'Context information is below.\n\n---------------------\n{context_str}\n---------------------\n\nGiven the context information and not prior knowledge.\ngenerate only questions based on the below query.\n\nYou are a Teacher/ Professor. Your task is to setup {num_questions_per_chunk} questions for an upcoming quiz/examination. The questions should be diverse in nature across the document. Restrict the questions to the context information provided."\n', num_questions_per_chunk: int = 2) -> EmbeddingQAFinetuneDataset
   Generate examples given a set of nodes.