Some are absolutely essential - sooner or later you'll want to sharpen your pencils or erase something - while others fall more in the category of 'luxuries.'
The Must HavesAt the very minimum, you'll need a good sharpener, eraser and a few sheets of paper.
SharpenerYou don't need an expensive sharpener for your pencils, anything with a sharp blade will get the job done. But some pencils (looking at you Prismacolor) can break very easily so you might want to get a suitable sharpener.
Or you might have a few oddly sized pencils that won't fit into a regular sharpener. Some people swear by electric and others will tell you that manual is the way to go.
What I have found is that certain types of sharpeners work best with certain pencils. Finding out which is the best fit for your pencils can be a trial and error process.
I personally prefer manual sharpness as I don't have to worry about changing batteries or charging yet another device in my home.
My favorite watercolor pencils will not fit into most regular sharpeners so I ended up buying the Dahle 133. It produces a nice long point and has an over-sized hole that can take pencils of any shape and size. The KUM brand of sharpness also have pretty good reviews and they are very cheap to boot.
EraserErasers also come in many variants - electric, kneaded, pencil-type erasers and so on. Electric erasers can be very handy for tiny spaces while kneaded erasers are useful for lifting off excess pigment. Any good quality eraser will work for colored pencils although it may not always be possible to completely erase your marks.
Just be careful not to rub too hard – you might end up erasing the print or even tear the paper. I use this Tombow eraser which is very useful for erasing in small details and spaces.
Sheets of PaperThankfully, you don't need any kind of special paper here. Just a few sheets of your average printer paper. I always keep a couple of sheets behind the page I am coloring to prevent bleed through or the ink transferring to the opposite page. It also helps prevent the indentations from ruining the next picture. After I have completed an image, I place a sheet of paper in between as well so that any excess pigment is not smeared on the opposite side.
The Nice to HaveBasically, that's all you need to get started - a book or a few sheets of artwork, medium (pencils or pens), a good sharpener and an eraser and a few sheets of paper to place in between.
But there are many other things that can enhance your coloring experience or the end results. None of these are essential to getting started and so I would recommend getting them as and when you feel the need.
Blending Pencils/MarkersAs you begin to use more advanced coloring techniques such as shading, layering, and blending, you might want to get some special tools. These tools help get a smoother finish on your artwork. Blending pencils and markers are available which can enhance the appearance of your colors and seamlessly blend transitions between colors.
Blending pencils are available from Caran D'ache, Prismacolor and Derwent to name a few. Colorless markers achieve the same thing for alcohol or water-based pens. People use tortillions or paper stumps to blend color pencil pigment. Many artists use odorless mineral spirits (such as Gamsol) or even Vaseline to blend color pencils. From my experience, Gamsol works best for any pencil.
PastelsAlthough not essential, soft pastels can be very useful for covering large areas such as the sky in landscapes. Quite a few artists color everything on a page with pencils but it can take a lot of time.
Soft pastels, on the other hand, give excellent results with minimal effort. You can get a high-quality set of pastels from Faber Castell for just ten dollars, and even the smallest 12 pack should be sufficient for colorists.
You don't have to buy everything before starting to color. Get each tool as and when you feel the need for it.