Construction of cryptosystems with public key (personal one way functions) is usually developed by using one of many mathematical problems. One of those problems is factoring big integers or testing if they are prime numbers or not. One example is RSA cryptosystem which uses big prime numbers p and q which are used to construct public module n. In this master thesis I will try to explain the way that RSA cryptosystem works to make it more clear what are his weaknesses. Power of RSA cryptosystem is in the problem of factoring large integers and testing if they are primes. Problem of factoring is larger if the number is larger than test can handle it. Methods which are used to test if number is prime or not are called primality tests. There have been shown theory and program implementation of Solovay-Strassen and Miller-Rabin test so it could be pointed where the problems are and what is main functionality of tests. Primality test are separated in two groups, deterministic and possibility ones. Probability tests are used more because of their fast way of testing. If one of probability tests does not work for even one example than we can say that the number is complex. Defining that the number is complex we do not get its factors. If the number is complex than it could be tested for its factors. In that case there have been implemented and explained some tests as factor bases, continued fraction method, quadratic sieve method, Pollard's (p-1) and Fermat’s factorization. With quadratic sieve method they have managed to factor RSA-640 number with 193 digits and results like that are making us aware of how complex the systems should be to remain safe, today but in a near future to.