Contributor
Mishra, Anshuman

Enhance Tokenizers and Implement end-to-end Token Classification Tasks


Mentors
Matthew Watson, Chen Qian
Organization
TensorFlow
Technologies
python, tensorflow
Topics
machine learning, natural language processing
The objective of this project is to implement Token Classification tasks for the KerasNLP library. The focus will be on enhancing the tokenizers API to allow for greater flexibility with offset mapping use cases. The ultimate goal is to use the enhanced tokenizer for the implementation of a Token Classification Pipeline. This will involve creating separate preprocessing and task classes for all the models. Finally, the project will be supplemented with a Keras-io example demonstrating Token Classification using the KerasNLP library.