FPGA stands for Field Programmable Gate Array. They are in essence (re-)configurable digital logic chips. To use an FPGA you design a "digital logic circuit" and then you upload that circuit to the FPGA chip. Therefore to fully understand what they are you must first understand what a digital circuit is.
Configuring an FPGA device is sometimes referred to as "FPGA programming". You see that term used everywhere; in articles, on book covers, etc. That is somewhat of a misnomer since the term programming has long been established to refer to programming a computer CPU w/ software. It is NOT the same thing as programming a CPU or a microcontroller. However since configuring a FPGA with a digital circuit using source code looks a lot like software programming and the process of uploading a circuit closely resembles compiling and running a software program, then the term has stuck. I even use the term myself. But for end users of the technology it can be confusing.
FPGA's are a blank slate, they are like Legos for digital electronics. They are indeed programmable hardware in the sense that you can program any digital circuit into them and they become that circuit.
FPGA's are configured using one of the HDL languages. The two industry standard ones are Verilog and VHDL. These are the languages that the tools for configuring an FPGA use. And the two most popular FPGA vendors are Intel (used to be Altera) and Xilinx. There are some higher abstraction HDL languages that then get converted to Verilog or VHDL. The one I am familiar with is MyHDL but there are others.
What can you do with a FPGA? You can do anything that can be done w/ a digital circuit. You can create dedicated circuits like a signal or digital image processor that performs many computations in parallel, or you can create a custom CPU or re-create the functionality of outdated and difficult to find hardware, such as retro computers, consoles or arcade games -- which is the primary use for retro tech hobbyists.