Since the built-in zoom uses an internal ChartToolset this kind of zooming (and panning in the same way) is lost when you attach your own toolset to a chart. In this case you should add a ZoomDragTool to the toolset, set its Shift to ssLeft; and to restore the default panning you should add a PanDragtool and set its Shift to ssRight.
There are also Chart.AllowZoom and AllowPanning property which you might have unchecked incidentally.