Clean up building of libffi for dynamic lib way