Class TokenizerFactory

    • Method Detail

      • availableTokenizers

        public static Set<StringavailableTokenizers()
        returns a list of all available tokenizer names from context classpath
      • reloadTokenizers

        public static void reloadTokenizers(ClassLoader classloader)
        Reloads the factory list from the given ClassLoader. Changes to the factories are visible after the method ends, all iterators (availableTokenizers(),...) stay consistent.

        NOTE: Only new factories are added, existing ones are never removed or replaced.

        This method is expensive and should only be called for discovery of new factories on the given classpath/classloader!

      • create

        public final Tokenizer create()
        Creates a TokenStream of the specified input using the default attribute factory.