This seems like a straightforward question but it's actually quite complex in its implications, and the answer is anything but simple. The trivial response is that ¿hackers get inside a target computer system by exploiting vulnerabilities,¿ but in order to provide more detail, let's start from the beginning.
The term ¿hacker¿ is fairly controversial in its meaning and interpretation. Some people claim that hackers are good guys who simply push the boundaries of knowledge without doing any harm (at least not on purpose), whereas ¿crackers¿ are the real bad guys. This debate is not productive; for the purposes of this discussion, the term ¿unauthorized user¿ (UU) will suffice. This moniker covers the entire spectrum of folks, from those involved in organized criminal activities to insiders who are pushing the limits of what they are authorized to do on a system.
Next let's explore what it means to ¿get inside¿ a computer. This can refer to gaining access to the stored contents of a computer system, gaining access to the processing capabilities of a system, or intercepting information being communicated between systems. Each of these attacks requires a different set of skills and targets a different set of vulnerabilities.
So what do UUs take advantage of? Vulnerabilities exist in every system and there are two kinds: known and unknown. Known vulnerabilities often exist as the result of needed capabilities. For instance, if you require different people to use a system in order to accomplish some business process, you have a known vulnerability: users. Another example of a known vulnerability is the ability to communicate over the Internet; enabling this capability, you open an access path to unknown and untrusted entities. Unknown vulnerabilities, which the owner or operator of a system is not aware of, may be the result of poor engineering, or may arise from unintended consequences of some of the needed capabilities.
By definition, vulnerabilities may be exploited. These can range from poor password protection to leaving a computer turned on and physically accessible to visitors to the office. More than one technical exploit has been managed simply by sitting at the receptionist's desk and using his computer to access the desired information. Poor passwords (for example, a username of ¿Joe Smith¿ with an accompanying password of ¿joesmith¿) are also a rich source of access: password cracking programs can easily identify dictionary words, names, and even common phrases within a matter of minutes. Attempts to make those passwords more complex by replacing letters with numbers, such as replacing the letter O with the number zero, don't make the task much harder. And when a UU can utilize a valid username-password combination, getting access to a system is as easy as logging in.
If a target system is very strongly protected (by an architecture that includes both technical controls such as firewalls or security software, and managerial controls such as well defined policies and procedures) and difficult to access remotely, a UU might employ low-technology attacks. These tactics may include bribing an authorized user, taking a temporary job with a janitorial services firm, or dumpster diving (rifling through trash in search of information). If the target system is not so strongly protected, then a UU can use technical exploits to gain access.
To employ technical exploits a UU must first determine the specifications of the target system. It would do no good whatsoever for a UU to use a technical exploit against a Microsoft vulnerability if the target system is a Macintosh. The UU must know what the target system is, how it is configured, and what kind of networking capabilities it has. Once these parameters (which can be determined remotely through a variety of methods) are known, then the UU can exploit the configuration's known vulnerabilities. The availability of preprogrammed attacks for common configurations can make this task quite simple; UUs that use these scripted capabilities are somewhat derisively known as ¿script kiddies.¿
One way a technically proficient UU can remotely determine the configuration of a target system is through capabilities inherent in hypertext transfer protocol (http). Users who access certain Web sites actually send configuration information, such as the type of browser being used, to the requesting site. Once the system configuration is known, then exploits can be selected. An example of an exploit that takes advantage of system-specific vulnerabilities is described in the following statement from the U.S. Computer Emergency Response Team (US CERT): ¿Exploit code has been publicly released that takes advantage of a buffer overflow vulnerability in the Microsoft Private Communication Technology (PCT) protocol. The vulnerability allows a remote attacker to execute arbitrary code with SYSTEM privileges.¿ [see http://www.us-cert.gov/current/current_activity.html for more information]
Another type of attack is one that is preprogrammed against specific vulnerabilities and is launched without any specific target--it is blasted out shotgun style with the goal of reaching as many potential targets as possible. This type of attack eliminates the need for the first step, but is less predictable in both outcome and effectiveness against any given target.
It's important to recognize that the end goal of unauthorized access varies depending on the UU's motivations. For example, if a UU is trying to gather a lot of zombie computers for use in a distributed denial of service attack, then the goal is to sneak a client program onto as many computers as possible. One way to do this fairly effectively is through the use of a so-called Trojan horse program, which installs the malicious program without the knowledge or consent of the user. Some of more recent mass Internet attacks have had this profile as an element of the attack pattern.
Protecting yourself against attacks is a multistep process, which aims to limit and manage the vulnerabilities of your system. (It's impossible to eliminate them all.) First, make sure you have all the latest patches for your operating system and applications--these patches generally fix exploitable vulnerabilities. Make sure your password is complex: it should include letters, numbers, and symbolic characters in a nonsensical manner. Also, consider getting a hardware firewall and limiting the flow of data to and from the Internet to only the few select ports you actually need, such as e-mail and Web traffic. Make sure your antivirus software is up-to-date and check frequently to see if there are new virus definitions available. (If you are using a Windows system, you should ideally update your virus definitions every day.) Finally, back up your data. That way if something bad does happen, you can at least recover the important stuff.