Because it was so large, corruption ran rampant and politicians and senators often had their own interests at heart. The sheer size of the empire also made it hard to control, with many revolutions and uprisings popping up. The cultural diffusion was also an issue, as many different people and cultures were all under one banner, so infighting was pretty common, epically the further you get away from Italy.