HALFT: High-Activation Layer Fine-Tuning.
A novel parameter-efficient fine-tuning method that dynamically selects layers based on activation-to-weight norm ratios. Experimental validation on Llama-3-8B, Qwen2.5-7B, and Gemma-3-12B demonstrates up to 6x training speedup and up to 80% data reduction compared to standard LoRA applied to all layers, while achieving 10-25% higher performance. Evidence from these three LLM families indicates the strategy is both LLM-agnostic and task-agnostic.Read White Paper ↗
A novel parameter-efficient fine-tuning method that dynamically selects layers based on activation-to-weight norm ratios. Experimental validation on Llama-3-8B, Qwen2.5-7B, and Gemma-3-12B demonstrates up to 6x training speedup and up to 80% data reduction compared to standard LoRA applied to all layers, while achieving 10-25% higher performance. Evidence from these three LLM families indicates the strategy is both LLM-agnostic and task-agnostic.Read White Paper ↗
Posts
Projects
Gaia
A no-code platform to build and deploy translation agents. Used by researchers, universities and governments.
View Project ↗Fino
A Heroku-clone, up to 10x less expensive using Vultr as a cloud provider. Built the backend and frontend from scratch.
View Project ↗VortiX
VortiX is an AI-powered academic search engine that provides answers based on +220 million papers
View Project ↗Athena
Free legal assistance for citizens in Peru. Ranked in the top 20 globally in the Digital Gamechangers Award by the United Nations Development Programme.
View Project ↗EduX
Get ready for the Duolingo English Test with our app. Trusted by +30K learners. [Discontinued]
View Project ↗Media Appearances
Book: Testimonies of the power of Education
16 Stories that Inspire (Me included). By Ministry of Education of Peru
View More ↗