Newark Airport is the furthest away of all airports serving New York city, and as a result, taxi rides are longer and more expensive. The train can be much quicker, especially in rush hour, and is a lot cheaper.
Train
Take the AirTrain to Newark Liberty International Airport Station (7-11 minutes). From there, the NJ Transit will take you to Penn Station (around 30 minutes) and to the World Trade Center in 40 minutes. The AirTrain runs every three minutes from 05:00-02:00 and every 15 minutes from 00:00-05:00. Journey time should be under an hour, including transfer times between trains.
Location: The AirTrain stops in the upper levels of Terminals A, B and C
Cost: USD 5.50 to the Airport Station, plus USD 13 to Penn Station
Bus
The Airport Express Bus, run by Coach USA, offers a direct service to New York City from all airport terminals. Buses run from 04:00-02:00 and leave every 15 minutes from 06:45-11:15. They will drop you off at Bryant Park, Grand Central Station and the Port Authority Bus Station. Journey time is around 45 minutes (longer in rush hour).
Cost: USD 16 one-way, USD 28 round-trip
Locations: Terminal A, arrivals, level 1, lane 5; Terminal B, arrivals, level 1, lane 2; Terminal C, arrivals, level 1, bus stops 5 and 6
Taxi
Taxis are available outside each terminal, and costs vary depending on traffic. Journey time can be as short as 30 minutes or as long as 60, depending on the traffic.
Cost: USD 50-70 (plus tolls and tip)
Going further afield?
Amtrak trains run from the Airport Station to Philadelphia, Boston and New Haven, Connecticut, among other destinations. It also runs hourly to New York city.