Options
All
  • Public
  • Public/Protected
  • All
Menu

A reusable class storing words embeddings for functions and class that needs it.

Hierarchy

  • KeyedVectors

Index

Constructors

constructor

  • new KeyedVectors(__namedParameters: { cased: boolean; loaderFunction: loaderFunction; maxDistance: number; size: number; tokenization: "word" | "byte_pair"; unknownKey: string }): KeyedVectors
  • Build a KeyedVector.

    Parameters

    • __namedParameters: { cased: boolean; loaderFunction: loaderFunction; maxDistance: number; size: number; tokenization: "word" | "byte_pair"; unknownKey: string }
      • cased: boolean
      • loaderFunction: loaderFunction
      • maxDistance: number
      • size: number
      • tokenization: "word" | "byte_pair"
      • unknownKey: string

    Returns KeyedVectors

Properties

Private cased

cased: boolean

Private loaderFunction

loaderFunction: () => Promise<string>

Type declaration

    • (): Promise<string>
    • Returns Promise<string>

Private maxDistance

maxDistance: number

Readonly size

size: number

Private tokenization

tokenization: "word" | "byte_pair"

Private trie

trie: Trie

Private unknownKey

unknownKey: string

Private vectors

vectors: {}

Type declaration

  • [key: string]: number[]

Methods

Private bytePairTokenize

  • bytePairTokenize(text: string): string[]
  • Tokenize a string based on the vocabulary.

    Parameters

    • text: string

      A non tokenized text string.

    Returns string[]

get

  • get(key: string): tf.Tensor1D
  • Return the vector associated with a key. If the key is not part of the vocabulary, it will use a similar key according to the leveinshtein distance. If no similar keys are below maxDistance, it will return the unknown key vector or undefined.

    Parameters

    • key: string

    Returns tf.Tensor1D

isLoaded

  • isLoaded(): boolean

keys

  • keys(): string[]

load

  • load(): Promise<void>

tokenize

  • tokenize(text: string): string[]
  • Tokenize a string based on the settings.

    Parameters

    • text: string

      A raw text string.

    Returns string[]

Private wordTokenize

  • wordTokenize(text: string): string[]
  • Tokenize a string at each non-word character.

    Parameters

    • text: string

      A non tokenized text string.

    Returns string[]

Legend

  • Class
  • Class with type parameter
  • Constructor
  • Property
  • Method
  • Function
  • Type alias
  • Private property
  • Private method
  • Enumeration
  • Interface

Generated using TypeDoc