Hashing algorithm used in Bitcoin is called SHA-256
The hashing algorithm is really hard to brute-force because it is very difficult to pick up a random input hash it and compare the output with the target hash as there are 2^128 computations involved and it is almost impossible.