For polymer-particle composites, limited thermodynamic compatibility of polymers and particles often leads to poor dispersal and agglomeration of the particles in the matrix, which negatively impacts the mechanics of composites. To study the impact of particle compatibility in polymer matrices on the mechanical properties of composites, we here study composite silica- protein based hydrogels. The polymer used is a previously studied telechelic protein-based polymer with end groups that form triple helices, and the particles are silica nanoparticles that only weakly associate with the polymer matrix. At 1mM protein polymer, up to 7% of silica nanoparticles can be embedded in the hydrogel. At higher concentrations the system phase separates. Oscillatory rheology shows that at high frequencies the particles strengthen the gels by acting as short-lived multivalent cross-links, while at low frequencies, the particles reduce the gel strength, presumably by sequestering part of the protein polymers in such a way that they can no longer contribute to the network strength. As is generally observed for polymer/particle composites, shear-induced polymer desorption from the particles leads to a viscous dissipation that strongly increases with increasing particle concentration. While linear rheological properties as function of particle concentration provide no signals for an approaching phase separation, this is very different for the non-linear rheology, especially fracture. Strain-at-break decreases rapidly with increasing particle concentration and vanishes as the phase boundary is approached, suggesting that the interfaces between regions of high and low particle densities in composites close to phase separation provide easy fracture planes.