Distilling Contextual Embeddings Into A Static Word Embedding For Improving Hacker Forum Analytics

Benjamin M. Ampel, Hsinchun Chen
Last updated on Jun 9, 2026
PDF DOI
In Plain Terms

This paper introduces Hack2Vec, a specialized way of converting hacker-forum text into numerical representations that machine-learning models can use. It transfers knowledge from the powerful BERT language model into a lightweight, forum-specific word embedding tailored to hacker language. On a standard forum classification task, Hack2Vec outperforms other widely used embedding methods across accuracy and related metrics.

Artifacts

PDF DOI

Related Papers

Cite this

APA Ampel, B. M., & Chen, H. (2021). Distilling Contextual Embeddings Into A Static Word Embedding For Improving Hacker Forum Analytics. In 2021 IEEE International Conference on Intelligence and Security Informatics (ISI). https://doi.org/10.1109/ISI53945.2021.9624848
BibTeX
@inproceedings{ampel2021distilling,
  title={Distilling Contextual Embeddings Into A Static Word Embedding For Improving Hacker Forum Analytics},
  author={Benjamin M. Ampel and Hsinchun Chen},
  booktitle={2021 IEEE International Conference on Intelligence and Security Informatics (ISI)},
  year={2021},
  doi={10.1109/ISI53945.2021.9624848}
}
Hacker Forums Cyber Threat Intelligence Deep Learning Exploit Labeling
Benjamin M. Ampel

Benjamin M. Ampel

Assistant Professor in Computer Information Systems and Director, Center for CyberAI Research (CCAIR)

My research focuses on AI-enabled Cybersecurity, including Cyber Threat Intelligence, Large Language Models, and Phishing Detection.